search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 63 Regressions on 9/16/2023 5:31:18 PM #22012

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.63 ns 6.20 ns 1.34 0.79 False
2.98 ns 4.76 ns 1.60 0.80 False
3.22 ns 7.22 ns 2.25 0.89 False
4.23 ns 11.60 ns 2.74 0.85 False
2.95 ns 4.77 ns 1.61 0.83 False
4.33 ns 6.75 ns 1.56 0.84 False
2.85 ns 5.23 ns 1.84 0.86 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.UnaryNegateOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.2009579411693405 > 4.89899143767928. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.096655252311866 (T) = (0 -7.785924317758165) / Math.Sqrt((1.2967078105025238 / (21)) + (4.973510655151477 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.44076268597196455 = (5.404029680644901 - 7.785924317758165) / 5.404029680644901 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.759568383878299 > 4.034004985938097. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.393877399651152 (T) = (0 -6.119433274075633) / Math.Sqrt((1.3451191724725382 / (18)) + (2.519968876773111 / (18))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (18) - 2, .025) and -0.4986234488432898 = (4.083369494050629 - 6.119433274075633) / 4.083369494050629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AbsBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.221870426811507 > 3.825976809173709. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.738376606024793 (T) = (0 -5.565509239686871) / Math.Sqrt((0.5592192981185623 / (16)) + (2.0295931828990588 / (21))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (21) - 2, .025) and -0.5975276208053961 = (3.4838266125758817 - 5.565509239686871) / 3.4838266125758817 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.600464393638257 > 3.2696430461094006. IsChangePoint: Marked as a change because one of 9/17/2023 8:50:51 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.84021339229518 (T) = (0 -6.267227893629676) / Math.Sqrt((1.184160451811741 / (22)) + (5.270484869509111 / (14))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (14) - 2, .025) and -0.6720694213212227 = (3.748186417210768 - 6.267227893629676) / 3.748186417210768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsStaticBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.768077666927842 > 3.4941263995706096. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.273514480043277 (T) = (0 -5.763083671221743) / Math.Sqrt((2.3004213182516766 / (20)) + (1.3417301879552845 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.4767852643895332 = (3.902452042412585 - 5.763083671221743) / 3.902452042412585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.DivideBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.751513777306751 > 3.130305870527888. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.8646046089556405 (T) = (0 -5.481995452080106) / Math.Sqrt((1.6293036504440372 / (18)) + (0.6756579285743511 / (20))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (20) - 2, .025) and -0.4552983376508273 = (3.766922087555777 - 5.481995452080106) / 3.766922087555777 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.231976092222044 > 3.557903726130916. IsChangePoint: Marked as a change because one of 9/18/2023 7:10:55 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -2.3521579207404555 (T) = (0 -5.933200769998546) / Math.Sqrt((2.7764337403106807 / (24)) + (5.950349492873815 / (12))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (12) - 2, .025) and -0.4493214349544066 = (4.093778389598713 - 5.933200769998546) / 4.093778389598713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.75 ns 6.78 ns 2.47 0.86 False
2.85 ns 5.09 ns 1.79 0.82 False
4.91 ns 9.44 ns 1.92 0.75 False
2.89 ns 4.95 ns 1.71 0.85 False
1.67 ns 3.30 ns 1.98 0.80 False
3.85 ns 7.43 ns 1.93 0.84 False
2.72 ns 5.09 ns 1.87 0.89 False
2.70 ns 7.73 ns 2.86 0.80 False
2.80 ns 5.12 ns 1.83 0.85 False
3.92 ns 10.05 ns 2.56 0.86 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AbsBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.779710443760463 > 3.1601977012015867. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.887983412424177 (T) = (0 -5.050147506161746) / Math.Sqrt((1.5038661105534743 / (19)) + (0.9934124537961075 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.38717621258974627 = (3.6405955208340313 - 5.050147506161746) / 3.6405955208340313 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.0911169485170635 > 3.119384554218085. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.878186265930022 (T) = (0 -5.067471016634662) / Math.Sqrt((0.9327041230207903 / (19)) + (0.624243153902861 / (20))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (20) - 2, .025) and -0.37512030351981485 = (3.685111043494706 - 5.067471016634662) / 3.685111043494706 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.UnaryNegateOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.441401477529547 > 4.98308675047997. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.917318903709294 (T) = (0 -7.2825782396973295) / Math.Sqrt((0.4737710883398102 / (19)) + (1.7560962304214245 / (17))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (17) - 2, .025) and -0.5154700773482378 = (4.8054912786139266 - 7.2825782396973295) / 4.8054912786139266 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivisionOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.946762705907901 > 3.1038147527376685. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.6839675884389305 (T) = (0 -5.257506839354559) / Math.Sqrt((1.4503718114792965 / (20)) + (0.5602002352881049 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.5273823472256973 = (3.442168130922932 - 5.257506839354559) / 3.442168130922932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ZeroBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.299298473532478 > 1.708461008802181. IsChangePoint: Marked as a change because one of 9/13/2023 5:14:37 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.9895734411069443 (T) = (0 -3.49074300733927) / Math.Sqrt((1.05540421831022 / (13)) + (1.4093423404358403 / (27))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (27) - 2, .025) and -0.7164607018767688 = (2.03368652921824 - 3.49074300733927) / 2.03368652921824 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SubtractBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.426459323225434 > 3.614506107753098. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.800665433037837 (T) = (0 -5.586862770748443) / Math.Sqrt((1.252231682048657 / (19)) + (1.1281494307835955 / (17))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (17) - 2, .025) and -0.4545635629103871 = (3.840920337348392 - 5.586862770748443) / 3.840920337348392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.XorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.089384374662026 > 3.186357483785959. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.009427743663345 (T) = (0 -5.339442280628585) / Math.Sqrt((1.2672827087526903 / (18)) + (1.2498428473358494 / (21))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (21) - 2, .025) and -0.3711049404325639 = (3.8942623012824007 - 5.339442280628585) / 3.8942623012824007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.727051614230993 > 3.156366166443373. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.589293182658719 (T) = (0 -5.9987472828244) / Math.Sqrt((0.7841374835669493 / (21)) + (1.2678181337858283 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.5639865055568758 = (3.835549259223612 - 5.9987472828244) / 3.835549259223612 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AddBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.122132708556004 > 4.391004572352039. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.187019197749541 (T) = (0 -5.192799365470122) / Math.Sqrt((1.4529860421157108 / (18)) + (0.502247961267898 / (21))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (21) - 2, .025) and -0.3528584441760466 = (3.8383907701686977 - 5.192799365470122) / 3.8383907701686977 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.DivideBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.046352613278438 > 3.3297353905627474. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.213906552907512 (T) = (0 -5.419641250794987) / Math.Sqrt((1.9306940481360786 / (21)) + (1.924934359276117 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.5192990771388056 = (3.5671984090199245 - 5.419641250794987) / 3.5671984090199245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.97 ns 4.88 ns 1.64 0.76 False
2.87 ns 4.89 ns 1.70 0.71 False
2.82 ns 4.09 ns 1.45 0.84 False
2.85 ns 7.00 ns 2.46 0.77 False
2.91 ns 5.16 ns 1.78 0.81 False
2.06 ns 4.88 ns 2.37 0.78 False
4.46 ns 6.42 ns 1.44 0.78 False
1.59 ns 2.94 ns 1.85 0.79 False
2.87 ns 5.05 ns 1.76 0.80 False
3.01 ns 4.95 ns 1.65 0.89 False
2.76 ns 5.08 ns 1.84 0.87 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.877690706738747 > 3.600411621145604. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.4018961512370054 (T) = (0 -5.417203592092001) / Math.Sqrt((3.0676449159276795 / (19)) + (0.7924179050111201 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.3948013626724787 = (3.8838530969832767 - 5.417203592092001) / 3.8838530969832767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.888309508138639 > 3.011618761671846. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.129051015964254 (T) = (0 -5.069571211105314) / Math.Sqrt((1.3122002019815173 / (18)) + (1.5326732948102386 / (21))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (21) - 2, .025) and -0.4515646115505437 = (3.492487465432255 - 5.069571211105314) / 3.492487465432255 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.087400866396905 > 3.0542939021436366. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.21634329266416 (T) = (0 -5.365039735741328) / Math.Sqrt((0.7338329413643971 / (19)) + (2.0122243221362446 / (17))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (17) - 2, .025) and -0.6266424908109233 = (3.298229184377642 - 5.365039735741328) / 3.298229184377642 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ExclusiveOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.001094946542092 > 4.256090054360209. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.2280472708588075 (T) = (0 -5.595651856268179) / Math.Sqrt((1.1228394169292089 / (21)) + (1.7762391176725263 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.5580279881865744 = (3.591496365082049 - 5.595651856268179) / 3.591496365082049 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.161513947120225 > 4.143829802705416. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.803570374491607 (T) = (0 -5.444208484027582) / Math.Sqrt((0.757048331338742 / (17)) + (1.0267250562314962 / (19))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (19) - 2, .025) and -0.5030431676693192 = (3.622123835917232 - 5.444208484027582) / 3.622123835917232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.879248217396119 > 4.148747587523253. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.787109820290155 (T) = (0 -5.348323318096392) / Math.Sqrt((0.8470703661882252 / (20)) + (0.8253339470611661 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.46398582156120816 = (3.6532616910134346 - 5.348323318096392) / 3.6532616910134346 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SubtractionOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.418362546054507 > 4.069005822141564. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.0717721158577564 (T) = (0 -5.251643186426332) / Math.Sqrt((1.700509128588596 / (21)) + (0.3814516931029597 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.327086034906129 = (3.9572740939872864 - 5.251643186426332) / 3.9572740939872864 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ZeroBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9419244407358587 > 1.7447857169484842. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.484138535034447 (T) = (0 -3.5188448816212006) / Math.Sqrt((0.4557371028105032 / (20)) + (0.9525258649704464 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.7266790356625539 = (2.037926452423142 - 3.5188448816212006) / 2.037926452423142 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.051342280366322 > 3.000649809380024. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.029826080311117 (T) = (0 -5.525534314020463) / Math.Sqrt((1.2342647911372455 / (20)) + (0.6547050644870321 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.5375175165780407 = (3.593802512454173 - 5.525534314020463) / 3.593802512454173 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MultiplyBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.949132197689485 > 3.9136019871895265. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.883611665994057 (T) = (0 -6.098889155599391) / Math.Sqrt((1.4406044336678243 / (20)) + (2.821345821945333 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.6207286703023037 = (3.7630537839882883 - 6.098889155599391) / 3.7630537839882883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.076491420829752 > 2.99415745084484. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.484769910985708 (T) = (0 -6.388853387008345) / Math.Sqrt((0.5431209738949297 / (21)) + (3.7790782031601995 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.9275548596701291 = (3.314485891261065 - 6.388853387008345) / 3.314485891261065 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.81 ns 5.00 ns 1.78 0.79 False
5.17 ns 6.79 ns 1.31 0.79 False
3.55 ns 5.07 ns 1.43 0.82 False
3.14 ns 5.39 ns 1.72 0.81 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.9976017829184745 > 3.71265585832082. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.3448894242635796 (T) = (0 -5.438695623325288) / Math.Sqrt((2.4952486376770704 / (20)) + (0.679609848117662 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.3269901475994564 = (4.098519972558925 - 5.438695623325288) / 4.098519972558925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.UnaryNegateOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.787386107218119 > 5.450562386411731. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.154180696144229 (T) = (0 -7.995306029435947) / Math.Sqrt((1.657771361793014 / (18)) + (3.3413077883574585 / (18))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (18) - 2, .025) and -0.3770617286672396 = (5.806062185152758 - 7.995306029435947) / 5.806062185152758 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.073081432343466 > 3.669130463428158. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.9759736840539146 (T) = (0 -5.531427930329723) / Math.Sqrt((1.7619429610446051 / (18)) + (1.0155091486497907 / (18))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (18) - 2, .025) and -0.3934438399853568 = (3.969609518233509 - 5.531427930329723) / 3.969609518233509 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.AddBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.39472224521062 > 3.761655968893365. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.577478681001329 (T) = (0 -5.2214570996113965) / Math.Sqrt((1.0764232913934761 / (20)) + (0.762679816266062 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.3674782614915898 = (3.8183108621529698 - 5.2214570996113965) / 3.8183108621529698 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
363.01 ns 383.76 ns 1.06 0.28 False
542.31 ns 606.87 ns 1.12 0.33 False
13.96 ns 15.97 ns 1.14 0.59 False
1.14 μs 1.25 μs 1.10 0.27 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 383.76352296853895 > 374.6082278821592. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.458298570985164 (T) = (0 -383.30092465322696) / Math.Sqrt((138.967184754167 / (21)) + (55.89806557968631 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.05495797706298593 = (363.3328843299908 - 383.30092465322696) / 363.3328843299908 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArrayIntArrayCtor(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 606.871595716119 > 571.6039343341457. IsChangePoint: Marked as a change because one of 9/17/2023 5:58:59 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.383456156061364 (T) = (0 -597.2462738869407) / Math.Sqrt((702.6270145894323 / (25)) + (19.929226352372368 / (14))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (14) - 2, .025) and -0.06166086970513184 = (562.5584317267163 - 597.2462738869407) / 562.5584317267163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 4) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.974364220756168 > 14.50264383894593. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.2504910852426105 (T) = (0 -16.437826321225923) / Math.Sqrt((0.9997123298958016 / (20)) + (2.0048825087154056 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.1506023922277956 = (14.286278589599501 - 16.437826321225923) / 14.286278589599501 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Collections.Tests.Perf_BitArray.BitArraySet(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.247735350750074 > 1.191993445436898. IsChangePoint: Marked as a change because one of 8/24/2023 4:35:49 AM, 9/15/2023 9:01:53 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -8.473255120348963 (T) = (0 -1239.8262760339344) / Math.Sqrt((1390.261339668193 / (16)) + (670.0486109679596 / (22))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (22) - 2, .025) and -0.0799525125109386 = (1148.0377717269087 - 1239.8262760339344) / 1148.0377717269087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.84 ns 4.95 ns 1.74 0.85 False
2.76 ns 5.39 ns 1.95 0.86 False
2.71 ns 7.40 ns 2.73 0.79 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MultiplyBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.952122783818174 > 3.3379593083660875. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.007917683323929 (T) = (0 -5.394224469708106) / Math.Sqrt((1.2070933752664548 / (20)) + (1.5957734437739375 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.4021526823909042 = (3.8471020577516946 - 5.394224469708106) / 3.8471020577516946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MultiplyOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.386133459541967 > 2.9015691580228498. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.344998418656241 (T) = (0 -5.847939297024691) / Math.Sqrt((1.5360267966097574 / (17)) + (1.9722530724854623 / (19))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (19) - 2, .025) and -0.674304462729512 = (3.492757396997657 - 5.847939297024691) / 3.492757396997657 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.MinBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.397584673103703 > 3.7996326040861064. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.2439720073260485 (T) = (0 -5.721322181885483) / Math.Sqrt((1.8652278305639332 / (21)) + (1.1019086711760333 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.7187086091460966 = (3.3288494346508215 - 5.721322181885483) / 3.3288494346508215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.34 ns 5.67 ns 2.42 0.84 False
6.35 ns 11.18 ns 1.76 0.66 False
3.80 ns 6.21 ns 1.63 0.73 False
3.31 ns 4.48 ns 1.35 0.80 False
5.74 ns 7.95 ns 1.39 0.80 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Constructors<String>.MemoryMarshalCreateSpan #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.6713499160898735 > 3.48918632606104. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.677991886358103 (T) = (0 -5.6547737566057465) / Math.Sqrt((0.8338416099445219 / (19)) + (0.8024512198256863 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.5303734752490569 = (3.695028597960687 - 5.6547737566057465) / 3.695028597960687 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<String>.SpanFromArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.17948130018282 > 6.653203506904681. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.671198417784771 (T) = (0 -9.240815944105949) / Math.Sqrt((2.3627774500588243 / (20)) + (2.3448693234667872 / (20))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (20) - 2, .025) and -0.3249379849021378 = (6.974527147237379 - 9.240815944105949) / 6.974527147237379 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<String>.ReadOnlySpanFromArray #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.212031491337235 > 4.165600529269462. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.257447075231103 (T) = (0 -7.3475401101667845) / Math.Sqrt((1.2582832836725162 / (22)) + (1.981292025142628 / (17))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (17) - 2, .025) and -0.5503451202725046 = (4.7392932154843725 - 7.3475401101667845) / 4.7392932154843725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<String>.MemoryMarshalCreateReadOnlySpan #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.478161206966383 > 3.514155193508818. IsChangePoint: Marked as a change because one of 9/12/2023 11:09:48 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.9290024743656358 (T) = (0 -4.66197737763585) / Math.Sqrt((0.6128237047661179 / (11)) + (0.47480942635665535 / (27))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (27) - 2, .025) and -0.2956186065300732 = (3.598263682027199 - 4.66197737763585) / 3.598263682027199 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<String>.SpanFromArrayStartLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.949071051134817 > 6.9536085087138115. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.9369766717491705 (T) = (0 -9.083313712110153) / Math.Sqrt((2.1645415191425146 / (21)) + (2.2208861863048552 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.2709156954871995 = (7.147062345963166 - 9.083313712110153) / 7.147062345963166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.73 ns 7.59 ns 1.60 0.81 False
1.40 ns 5.49 ns 3.93 0.86 False
2.89 ns 5.01 ns 1.73 0.83 False
2.97 ns 4.75 ns 1.60 0.85 False
2.97 ns 5.46 ns 1.84 0.87 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.590628494243033 > 4.809728569795888. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.688500960452226 (T) = (0 -5.781499939220497) / Math.Sqrt((1.4646270845117833 / (20)) + (1.5011810519566302 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.4726359852692464 = (3.925953186702448 - 5.781499939220497) / 3.925953186702448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.OneBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.491226486381241 > 1.5697249106756896. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.1742105450559315 (T) = (0 -3.844850002172674) / Math.Sqrt((1.1066094998024691 / (17)) + (0.9477419225546515 / (19))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (19) - 2, .025) and -0.839311801926322 = (2.090374235703778 - 3.844850002172674) / 2.090374235703778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.AddBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.01458913789146 > 3.0740995664448714. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.927375367932052 (T) = (0 -5.521868211454859) / Math.Sqrt((1.9249371674252553 / (19)) + (1.2334956571885813 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.4084207669222731 = (3.920609764595722 - 5.521868211454859) / 3.920609764595722 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.BitwiseAndBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.74972578092105 > 4.299154014256625. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.154253464891003 (T) = (0 -5.519297448840615) / Math.Sqrt((2.3757458651931946 / (19)) + (1.755245041506089 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.5407407114044711 = (3.5822363931757653 - 5.519297448840615) / 3.5822363931757653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int32>.ConditionalSelectBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.458991978621272 > 3.220955874175535. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.253717538863308 (T) = (0 -6.375458475889315) / Math.Sqrt((2.434110099083687 / (20)) + (1.699314680605146 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.4421301771187482 = (4.420861983920849 - 6.375458475889315) / 4.420861983920849 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.87 ns 3.24 ns 3.71 0.84 False
4.25 ns 8.13 ns 1.91 0.87 False
2.93 ns 5.17 ns 1.76 0.84 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.ZeroBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2431775837073746 > 1.7116985736154795. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.2127816121231456 (T) = (0 -3.496065641257973) / Math.Sqrt((1.7829135373046043 / (20)) + (0.23423415429571623 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.4139024747986119 = (2.472635633342344 - 3.496065641257973) / 2.472635633342344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.127030984003406 > 3.301367043479398. IsChangePoint: Marked as a change because one of 9/15/2023 3:29:05 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.715628546177322 (T) = (0 -5.468897754590725) / Math.Sqrt((0.7361275479014533 / (17)) + (1.921405567248091 / (23))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (23) - 2, .025) and -0.5929023293548396 = (3.4332913285435076 - 5.468897754590725) / 3.4332913285435076 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<SByte>.SubtractBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.1668394902037935 > 3.7548423497484458. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.8046295378229877 (T) = (0 -5.5988977431787745) / Math.Sqrt((2.3589300883531714 / (21)) + (0.7562036334031793 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.36418746689332704 = (4.104199663942948 - 5.5988977431787745) / 4.104199663942948 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.MemoryMarshal<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.01 ns 6.23 ns 1.24 0.68 False
5.32 ns 6.48 ns 1.22 0.81 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.MemoryMarshal&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.MemoryMarshal<Int32>.CastToByte #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.229212551856006 > 5.401433695674287. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.371590899627199 (T) = (0 -7.101816008678614) / Math.Sqrt((0.8579529362203874 / (19)) + (1.2373639106803627 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.2569407257608544 = (5.650080280738557 - 7.101816008678614) / 5.650080280738557 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.MemoryMarshal<Int32>.AsBytes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.478525027226583 > 6.374825579464995. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.4671620004015016 (T) = (0 -7.826152119909861) / Math.Sqrt((1.439113221927057 / (21)) + (2.587046401456007 / (16))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (16) - 2, .025) and -0.26995022748421404 = (6.162566020727882 - 7.826152119909861) / 6.162566020727882 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.MemoryMarshal<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.80 ns 6.56 ns 1.73 0.80 False
5.19 ns 8.53 ns 1.64 0.81 False
4.38 ns 6.29 ns 1.44 0.80 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.MemoryMarshal&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.MemoryMarshal<Byte>.GetReference #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.555768351021257 > 5.30010089264195. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.454524763477075 (T) = (0 -6.065672014754255) / Math.Sqrt((0.8130307452829978 / (21)) + (1.4666008704064748 / (18))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (18) - 2, .025) and -0.34156651886408 = (4.521335266990809 - 6.065672014754255) / 4.521335266990809 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.MemoryMarshal<Byte>.CastToInt #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.52610212823598 > 6.683709211378806. IsChangePoint: Marked as a change because one of 9/17/2023 10:41:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.628063003540532 (T) = (0 -8.129721583991977) / Math.Sqrt((1.2335758030483301 / (26)) + (0.8251602970549672 / (13))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (13) - 2, .025) and -0.2996487088668402 = (6.255322325584624 - 8.129721583991977) / 6.255322325584624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.MemoryMarshal<Byte>.CastToByte #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.291579585126769 > 4.7163518281102155. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.929472817350518 (T) = (0 -7.7627314443139825) / Math.Sqrt((2.2853554264648976 / (20)) + (2.0881311184547284 / (17))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (17) - 2, .025) and -0.447613852295011 = (5.362432413869998 - 7.7627314443139825) / 5.362432413869998 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.13 μs 1.72 μs 1.53 0.22 False
793.21 ns 904.24 ns 1.14 0.28 False
1.14 μs 1.58 μs 1.39 0.25 False
1.36 μs 2.03 μs 1.50 0.24 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Span.IndexerBench.CoveredIndex1(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.717754011200915 > 1.1814697980928572. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.8110494274548605 (T) = (0 -1631.2984680277439) / Math.Sqrt((49175.82976439722 / (20)) + (51262.33847589195 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.34373664938423965 = (1214.0016191232694 - 1631.2984680277439) / 1214.0016191232694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Span.IndexerBench.SameIndex1(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 904.2392120517377 > 833.4346590773788. IsChangePoint: Marked as a change because one of 9/17/2023 10:41:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -8.213361719867354 (T) = (0 -905.3836158516973) / Math.Sqrt((2588.533628653044 / (26)) + (1.550306206416859 / (12))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (12) - 2, .025) and -0.09959635367017267 = (823.3781540196612 - 905.3836158516973) / 823.3781540196612 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Span.IndexerBench.SameIndex2(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5816603595656915 > 1.196574924801973. IsChangePoint: Marked as a change because one of 9/17/2023 10:41:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -12.234291113597799 (T) = (0 -1584.9942617499075) / Math.Sqrt((25851.798448593298 / (27)) + (6.465302631108056 / (12))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (12) - 2, .025) and -0.31390737610322494 = (1206.3211536650854 - 1584.9942617499075) / 1206.3211536650854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Span.IndexerBench.CoveredIndex3(length: 1024) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0340042063376242 > 1.4332628691505864. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.837677220579285 (T) = (0 -2002.972290225886) / Math.Sqrt((90340.34821346024 / (16)) + (20245.133042135465 / (22))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (22) - 2, .025) and -0.30919887285864484 = (1529.9221010268532 - 2002.972290225886) / 1529.9221010268532 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
38.56 ns 44.17 ns 1.15 0.44 False
38.71 ns 44.86 ns 1.16 0.50 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
### Payloads [Baseline]() [Compare]() ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.16583613234065 > 40.277739110492256. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.277961679199089 (T) = (0 -44.061944133787335) / Math.Sqrt((10.902917986260375 / (19)) + (7.6270106520728564 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.10604929632657675 = (39.83723354838374 - 44.061944133787335) / 39.83723354838374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.85920618493038 > 40.38848797708106. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.64970539197341 (T) = (0 -43.408161796052575) / Math.Sqrt((11.871775487704793 / (16)) + (3.3179803725845343 / (22))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (22) - 2, .025) and -0.11260940140513438 = (39.014735756530214 - 43.408161796052575) / 39.014735756530214 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)