search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 24 Regressions on 9/16/2023 5:31:18 PM #21888

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.59 ns 3.37 ns 2.11 0.82 False
2.83 ns 6.65 ns 2.35 0.81 False
3.01 ns 5.21 ns 1.73 0.85 False
2.93 ns 6.66 ns 2.27 0.82 False
2.74 ns 5.45 ns 1.99 0.87 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ZeroBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3697224683413807 > 2.6545907868003105. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.084842515979841 (T) = (0 -4.031714805248039) / Math.Sqrt((0.9151916813073807 / (30)) + (2.0903270717376263 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.7823989090239579 = (2.26195987039501 - 4.031714805248039) / 2.26195987039501 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.647852628270398 > 3.8580585031562165. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -4.3542914869822535 (T) = (0 -5.757296599224264) / Math.Sqrt((0.7917375681739864 / (29)) + (1.4624991749284622 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.5812579609634803 = (3.6409597556848166 - 5.757296599224264) / 3.6409597556848166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsStaticBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.212114850650349 > 3.085898562703746. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.2284507937956595 (T) = (0 -5.930719132033991) / Math.Sqrt((0.7725344578483275 / (30)) + (3.4919996979381174 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.6509268906083978 = (3.592357218100923 - 5.930719132033991) / 3.592357218100923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MaxBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.65563752991889 > 4.862626361102998. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.0524501542337084 (T) = (0 -6.2291105302404315) / Math.Sqrt((1.2313753480282457 / (30)) + (3.973582132792485 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.6189551191159064 = (3.847611621032509 - 6.2291105302404315) / 3.847611621032509 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.MinBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.454431484774413 > 3.1211713925170117. IsChangePoint: Marked as a change because one of 9/16/2023 6:40:29 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -7.435354993720014 (T) = (0 -5.418630449384581) / Math.Sqrt((1.203117561636619 / (30)) + (0.23979279528619074 / (8))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (8) - 2, .025) and -0.5704698242931288 = (3.4503244605947883 - 5.418630449384581) / 3.4503244605947883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.88 μs 5.15 μs 1.05 0.16 False
4.67 μs 5.18 μs 1.11 0.15 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### System.Tests.Perf_Random.NextBytes_span #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.1485317620064786 > 5.099084482080692. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -4.791866468965362 (T) = (0 -5273.920480072133) / Math.Sqrt((3660.666783314019 / (31)) + (50010.04272063534 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.08392886123568356 = (4865.559603293375 - 5273.920480072133) / 4865.559603293375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Tests.Perf_Random.NextBytes #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.180827860552699 > 4.960918354416964. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.4777264289114544 (T) = (0 -5261.502655056762) / Math.Sqrt((73045.23671182389 / (29)) + (34748.62809757625 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.09897188359501322 = (4787.659023491178 - 5261.502655056762) / 4787.659023491178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.03 ns 5.23 ns 1.73 0.88 False
1.34 ns 4.90 ns 3.66 0.78 False
2.92 ns 5.40 ns 1.85 0.81 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SquareRootBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.229350077030501 > 3.103046477941418. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -6.163301175489074 (T) = (0 -5.678914171486597) / Math.Sqrt((0.7973558385880011 / (30)) + (0.6550377151636865 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.6030797807161501 = (3.542502525326365 - 5.678914171486597) / 3.542502525326365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.NegateBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.903702564562991 > 2.9609180791949172. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -4.9239504992204095 (T) = (0 -5.010438257071743) / Math.Sqrt((0.7648350246695289 / (28)) + (1.3566249323965267 / (9))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (9) - 2, .025) and -0.7084612074985658 = (2.932719944169964 - 5.010438257071743) / 2.932719944169964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.399844533291463 > 3.0992702312759217. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.107283526548958 (T) = (0 -6.0916372993111265) / Math.Sqrt((0.7325284735547826 / (31)) + (1.1141281791258704 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.6222780706418948 = (3.7549896097040985 - 6.0916372993111265) / 3.7549896097040985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.35 ns 6.87 ns 1.58 0.81 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.UnaryNegateOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.8650271522207795 > 5.887548972198676. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.665975547657209 (T) = (0 -8.140146288276924) / Math.Sqrt((1.635736648603665 / (31)) + (2.536215482814572 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.4504113377563467 = (5.612301887317693 - 8.140146288276924) / 5.612301887317693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.30 ns 6.76 ns 2.94 0.75 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Collections.IterateFor<Int32>.Span(Size: 512) #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.758960118260457 > 4.828702513301549. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -10.996639067157062 (T) = (0 -6.529087232407154) / Math.Sqrt((0.2992805772855971 / (29)) + (0.13873638503657343 / (6))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (6) - 2, .025) and -0.44509822644955926 = (4.51809234341694 - 6.529087232407154) / 4.51809234341694 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.34 ns 5.64 ns 1.30 0.85 False
2.76 ns 8.76 ns 3.17 0.87 False
2.81 ns 5.08 ns 1.81 0.83 False
2.81 ns 5.89 ns 2.10 0.90 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Int16>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.640881433503313 > 3.4857756760390024. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -4.615079385595278 (T) = (0 -5.721304633964951) / Math.Sqrt((0.9278003353868225 / (29)) + (1.234222307171059 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.5826858986220925 = (3.6149337268664583 - 5.721304633964951) / 3.6149337268664583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.AndNotBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.75687567161967 > 3.171055760548121. IsChangePoint: Marked as a change because one of 9/16/2023 7:32:07 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -2.7476134107572263 (T) = (0 -6.709894666917133) / Math.Sqrt((2.1859500383915855 / (30)) + (4.3301871985032365 / (4))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (4) - 2, .025) and -0.7862173639805213 = (3.756482722777016 - 6.709894666917133) / 3.756482722777016 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.MinBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.0814087779501325 > 3.4931949407765313. IsChangePoint: Marked as a change because one of 9/15/2023 9:01:53 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.044573293383037 (T) = (0 -5.934609668721537) / Math.Sqrt((0.8699509418669698 / (28)) + (1.9906601538470703 / (10))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (10) - 2, .025) and -0.6885572509939538 = (3.5146037631997276 - 5.934609668721537) / 3.5146037631997276 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Numerics.Tests.Perf_VectorOf<Int16>.ExclusiveOrOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.886460622934728 > 3.4234063217482134. IsChangePoint: Marked as a change because one of 9/16/2023 1:27:38 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -6.771943100845733 (T) = (0 -6.003590979191447) / Math.Sqrt((1.200747271372713 / (29)) + (0.7581105008325244 / (9))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (9) - 2, .025) and -0.6661691792924661 = (3.6032301244108083 - 6.003590979191447) / 3.6032301244108083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.76 ns 7.91 ns 4.49 0.84 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Double>.OneBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.906901818467 > 1.8010755973454398. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.159035379797488 (T) = (0 -4.253317706597633) / Math.Sqrt((0.7778309245010899 / (31)) + (2.9002186659756255 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.9698032264820771 = (2.1592601988949647 - 4.253317706597633) / 2.1592601988949647 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.01 ns 5.10 ns 1.69 0.84 False
2.89 ns 8.76 ns 3.03 0.87 False
3.18 ns 5.06 ns 1.59 0.82 False
2.89 ns 7.75 ns 2.68 0.83 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseOrBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.102850529237321 > 3.4178437321924138. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.5879534461716616 (T) = (0 -5.713433881288451) / Math.Sqrt((0.9163420344076184 / (31)) + (2.116260964783319 / (7))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (7) - 2, .025) and -0.566852015685607 = (3.646441287429704 - 5.713433881288451) / 3.646441287429704 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AndNotBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.75632915246551 > 4.847596059353714. IsChangePoint: Marked as a change because one of 9/15/2023 4:40:29 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -3.3981185988763025 (T) = (0 -5.979830319104084) / Math.Sqrt((1.901107093630388 / (24)) + (2.648300470190332 / (10))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (10) - 2, .025) and -0.4999606102344681 = (3.9866582350914803 - 5.979830319104084) / 3.9866582350914803 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SubtractionOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.059881985537456 > 3.078830249238456. IsChangePoint: Marked as a change because one of 9/14/2023 11:50:12 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.126249625595875 (T) = (0 -5.158537970354859) / Math.Sqrt((1.0797462416265722 / (25)) + (0.9527235138878942 / (13))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (13) - 2, .025) and -0.5132027505883333 = (3.4090196890993085 - 5.158537970354859) / 3.4090196890993085 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.BitwiseAndOperatorBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.748343346262252 > 3.5990418520819123. IsChangePoint: Marked as a change because one of 9/16/2023 5:31:18 PM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.227313582550945 (T) = (0 -6.3244995918764815) / Math.Sqrt((2.6983850371398947 / (30)) + (0.7659721573390759 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.6274935909809093 = (3.886036557639918 - 6.3244995918764815) / 3.886036557639918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.Constructors_ValueTypesOnly<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.52 ns 5.68 ns 1.61 0.89 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors_ValueTypesOnly&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Memory.Constructors_ValueTypesOnly<Byte>.ReadOnlyFromPointerLength #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.683250983181308 > 4.218756438520917. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -6.803486337490796 (T) = (0 -5.785101224368537) / Math.Sqrt((0.7859279183801605 / (29)) + (0.17726574963626843 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.3684948881294296 = (4.227345877978459 - 5.785101224368537) / 4.227345877978459 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.71 ns 7.26 ns 1.95 0.88 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### System.Numerics.Tests.Perf_VectorOf<Single>.MultiplyBenchmark #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.255153902390808 > 4.546680496688626. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -2.6950166843119256 (T) = (0 -6.58252458383346) / Math.Sqrt((1.5348188159089846 / (29)) + (3.574640711850627 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.44375497185186735 = (4.559308686147917 - 6.58252458383346) / 4.559308686147917 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline fd20f42f1f4fa77a0facde02e0c1527fa26e95cb
Compare f5cff047f77be26cfbbd9126e9e0d1242baac24d
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.85 ns 9.27 ns 1.91 0.71 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### System.Memory.ReadOnlySpan.StringAsSpan #### ETL Files #### Histogram #### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.267469625801045 > 5.360608472461216. IsChangePoint: Marked as a change because one of 9/16/2023 11:57:54 AM, 9/18/2023 8:11:22 PM falls between 9/9/2023 9:29:58 PM and 9/18/2023 8:11:22 PM. IsRegressionStdDev: Marked as regression because -5.463682526389245 (T) = (0 -7.55869411659553) / Math.Sqrt((0.5783025003695311 / (30)) + (1.050010594497789 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.4675751522468128 = (5.150464768378914 - 7.55869411659553) / 5.150464768378914 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

the range https://github.com/dotnet/runtime/compare/fa1bf3c0f9ec4e204203a49191170f9d53cd549a...dc3d344c4803aaa74c2df444632b95554762bfa1 contains only docs change. infra change?