search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 16 Regressions on 9/17/2023 8:50:51 AM #22019

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline f5cff047f77be26cfbbd9126e9e0d1242baac24d
Compare 51a694bfb475dd14992d050bab99888ce2514a38
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.69 ns 4.36 ns 1.62 0.78 False
1.14 ns 7.12 ns 6.24 0.87 False
2.64 ns 4.65 ns 1.76 0.89 False
2.95 ns 4.44 ns 1.50 0.84 False
2.63 ns 4.86 ns 1.85 0.78 False
2.48 ns 6.96 ns 2.81 0.84 False
1.90 ns 6.37 ns 3.35 0.83 False
2.85 ns 6.43 ns 2.26 0.87 False
3.01 ns 9.57 ns 3.18 0.81 False
2.98 ns 7.46 ns 2.50 0.82 False
3.92 ns 6.20 ns 1.58 0.84 False
2.94 ns 5.14 ns 1.75 0.86 False
2.84 ns 5.22 ns 1.84 0.78 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int16>.UnaryNegateOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.355990316264853 > 3.0961877765599386. IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.768119796265714 (T) = (0 -4.967681530946645) / Math.Sqrt((0.9704467269846936 / (23)) + (1.4550339500860725 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.3826795874073347 = (3.592792991369428 - 4.967681530946645) / 3.592792991369428 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.XorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.119418877914863 > 4.572401490644199. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.693358641717963 (T) = (0 -5.610837222314069) / Math.Sqrt((1.5146027236275736 / (23)) + (1.4698752098618049 / (17))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (17) - 2, .025) and -0.48468984022685424 = (3.7791308799262455 - 5.610837222314069) / 3.7791308799262455 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.652235685131807 > 2.8471456331078797. IsChangePoint: Marked as a change because one of 9/18/2023 8:11:22 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.7930510660379273 (T) = (0 -5.449374499964152) / Math.Sqrt((1.1597016846094503 / (27)) + (2.2772452705016484 / (9))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (9) - 2, .025) and -0.6094771346636838 = (3.38580423579789 - 5.449374499964152) / 3.38580423579789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.AddBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.435567089401261 > 3.216118465344543. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.456608038759147 (T) = (0 -5.585996666942416) / Math.Sqrt((1.596714590223351 / (20)) + (1.17909556017868 / (18))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (18) - 2, .025) and -0.4371059003647885 = (3.886976363763096 - 5.585996666942416) / 3.886976363763096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.NegateBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.8639196964205045 > 2.860037157853803. IsChangePoint: Marked as a change because one of 9/12/2023 11:09:48 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.612788742755408 (T) = (0 -4.828409853266537) / Math.Sqrt((0.4480813257129439 / (12)) + (1.3731121908516652 / (26))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (26) - 2, .025) and -0.5361686809724182 = (3.143150822610236 - 4.828409853266537) / 3.143150822610236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.962663058567798 > 3.8417913868180387. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.512102548117161 (T) = (0 -5.68850104043744) / Math.Sqrt((1.1374915544175872 / (20)) + (0.934190298216016 / (16))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (16) - 2, .025) and -0.4902551386318868 = (3.817132310417469 - 5.68850104043744) / 3.817132310417469 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.AbsBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.369502958079877 > 3.2302221556566026. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.915274944470484 (T) = (0 -4.9904791634592485) / Math.Sqrt((0.980020638176547 / (21)) + (0.613058292087307 / (19))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (19) - 2, .025) and -0.4992851360800367 = (3.328572426527972 - 4.9904791634592485) / 3.328572426527972 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.MultiplyBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.430118620894064 > 4.294999645970952. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.073337304971306 (T) = (0 -6.142153567592504) / Math.Sqrt((1.2801208776494424 / (22)) + (1.8716967611984734 / (16))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (16) - 2, .025) and -0.5283537352081756 = (4.018803648721994 - 6.142153567592504) / 4.018803648721994 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.567583415977428 > 3.0686502288392714. IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.804780066094999 (T) = (0 -6.487006109764661) / Math.Sqrt((0.9378480797978274 / (23)) + (4.285470966536198 / (16))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (16) - 2, .025) and -0.699153137998548 = (3.817787793633351 - 6.487006109764661) / 3.817787793633351 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.MinBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.457551168614413 > 3.3942234496419306. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -7.1064630020612505 (T) = (0 -5.584908319964227) / Math.Sqrt((0.5419742389696853 / (18)) + (1.4104986526469478 / (22))) is less than -2.0243941639098457 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (22) - 2, .025) and -0.6409202109443928 = (3.4035221717148363 - 5.584908319964227) / 3.4035221717148363 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.200015919635736 > 3.554160611673996. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.893011032430943 (T) = (0 -5.728837472652991) / Math.Sqrt((0.8907849154648607 / (19)) + (1.3317875530507395 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.6993004541788963 = (3.3712916739148233 - 5.728837472652991) / 3.3712916739148233 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.SubtractBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.143907976085514 > 3.6280879619056905. IsChangePoint: Marked as a change because one of 9/18/2023 2:28:59 PM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.75214360465792 (T) = (0 -6.36164788846747) / Math.Sqrt((1.4042292064622814 / (28)) + (3.05412134809485 / (11))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (11) - 2, .025) and -0.5098582313822075 = (4.21340742875155 - 6.36164788846747) / 4.21340742875155 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.221823065965114 > 3.0108015193856703. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -6.890062709135014 (T) = (0 -5.419930746567598) / Math.Sqrt((0.9144291090298603 / (19)) + (1.0328891402148477 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.6862785350438702 = (3.2141373052741806 - 5.419930746567598) / 3.2141373052741806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline f5cff047f77be26cfbbd9126e9e0d1242baac24d
Compare 51a694bfb475dd14992d050bab99888ce2514a38
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.08 ns 8.26 ns 1.17 0.66 False
5.00 ns 6.19 ns 1.24 0.71 False
12.83 ns 17.70 ns 1.38 0.63 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Constructors<Byte>.SpanFromMemory #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.255550079639296 > 8.109551931927674. IsChangePoint: Marked as a change because one of 9/17/2023 1:05:21 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -3.6496581182260015 (T) = (0 -9.609277678226977) / Math.Sqrt((0.6275239051065992 / (22)) + (1.710253487573167 / (16))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (16) - 2, .025) and -0.16247123967824986 = (8.266249822134649 - 9.609277678226977) / 8.266249822134649 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<Byte>.ReadOnlySpanFromArrayStartLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.186913995806405 > 5.236851165379159. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -4.585720300830494 (T) = (0 -7.274475648748187) / Math.Sqrt((1.3676597577478973 / (20)) + (1.423102843169075 / (19))) is less than -2.026192463026769 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (19) - 2, .025) and -0.3134013542895139 = (5.538653987975613 - 7.274475648748187) / 5.538653987975613 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Memory.Constructors<Byte>.MemoryFromArrayStartLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.698959261099947 > 13.890027585188381. IsChangePoint: Marked as a change because one of 8/22/2023 4:48:44 PM, 9/16/2023 11:57:54 AM, 9/20/2023 11:58:56 PM falls between 9/11/2023 8:56:48 PM and 9/20/2023 11:58:56 PM. IsRegressionStdDev: Marked as regression because -5.771269533989577 (T) = (0 -15.875245405121046) / Math.Sqrt((1.3512392178208104 / (19)) + (2.525732450160265 / (19))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (19) - 2, .025) and -0.19648406912710012 = (13.26824636846431 - 15.875245405121046) / 13.26824636846431 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

range is probably https://github.com/dotnet/runtime/compare/a7511515caa33e51edcb291dd3db2008947d2f51...9f4c5f7766ba6e6899bc998e17ca91a4fcc2d8d5

nothing looks related, values got back to normal in this case https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu%2022.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).MinBenchmark.html