performanceautofiler[bot]
commented
2 years ago Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | af88ee90bacedb49b90f7dc62a3092b287fc6ed0 |
| Compare | 789b420913a93f243adb4a6b14b162dc99c01242 |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_VectorOf<Double>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).OneBenchmark.html>) | 34.66 ns | 38.49 ns | 1.11 | 0.21 | False | |||||
| [MaxBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).MaxBenchmark.html>) | 58.27 ns | 62.34 ns | 1.07 | 0.19 | False | |||||
| [LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).LessThanOrEqualBenchmark.html>) | 56.57 ns | 61.78 ns | 1.09 | 0.27 | False | |||||
| [AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).AndNotBenchmark.html>) | 83.73 ns | 88.53 ns | 1.06 | 0.29 | False | |||||
| [MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).MultiplyBenchmark.html>) | 55.08 ns | 61.49 ns | 1.12 | 0.24 | False | |||||
| [SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).SquareRootBenchmark.html>) | 35.35 ns | 38.44 ns | 1.09 | 0.21 | False | |||||
| [DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).DivideBenchmark.html>) | 56.04 ns | 61.91 ns | 1.10 | 0.22 | False | |||||
| [OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_VectorOf(Double).OnesComplementBenchmark.html>) | 60.62 ns | 64.71 ns | 1.07 | 0.19 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf<Double>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Numerics.Tests.Perf_VectorOf<Double>.OneBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.48547492244615 > 36.437204675782716.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.22652502412602 (T) = (0 -38.866738279654) / Math.Sqrt((0.8650238963454278 / (35)) + (0.7138671586396629 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.10818026794719117 = (35.07257745317131 - 38.866738279654) / 35.07257745317131 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.MaxBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.341376186331125 > 61.398367734998565.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.526485542753173 (T) = (0 -62.43185769274923) / Math.Sqrt((6.3767971085968815 / (35)) + (4.365339465037886 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.053807860466580364 = (59.24406149818157 - 62.43185769274923) / 59.24406149818157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.77626692891586 > 59.32879982365056.
IsChangePoint: Marked as a change because one of 7/6/2022 12:32:56 PM, 7/7/2022 1:00:04 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -10.22261491467451 (T) = (0 -62.10827326245255) / Math.Sqrt((1.5729644122168314 / (35)) + (6.845692460512102 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.094115625328189 = (56.76573099284882 - 62.10827326245255) / 56.76573099284882 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.AndNotBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 88.52881760057112 > 88.25894660627846.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 11:19:50 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.976930503355394 (T) = (0 -89.6600879607903) / Math.Sqrt((9.174689439498968 / (33)) + (14.182841213611896 / (31))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (31) - 2, .025) and -0.060638313313575345 = (84.5340837072727 - 89.6600879607903) / 84.5340837072727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.MultiplyBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.48585851137682 > 58.07436209575573.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -5.955856257283941 (T) = (0 -60.35372995243013) / Math.Sqrt((5.75305272713782 / (35)) + (12.115476835621358 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.08254651681132534 = (55.7516273113177 - 60.35372995243013) / 55.7516273113177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.SquareRootBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.43862020884357 > 37.02715693967304.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -7.571803790044796 (T) = (0 -39.09720246008806) / Math.Sqrt((0.6733158458005493 / (35)) + (5.9358834305574435 / (27))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (27) - 2, .025) and -0.10460087942329568 = (35.3948681269387 - 39.09720246008806) / 35.3948681269387 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.DivideBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 61.91453747902374 > 59.04533209015445.
IsChangePoint: Marked as a change because one of 9/27/2022 4:04:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.968352647970752 (T) = (0 -59.79498890806816) / Math.Sqrt((3.5950896942478283 / (27)) + (9.263371295025019 / (37))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (37) - 2, .025) and -0.05424755785634824 = (56.71816686931878 - 59.79498890806816) / 56.71816686931878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.71413367785841 > 62.994042557395254.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.585426715304845 (T) = (0 -65.24126813954976) / Math.Sqrt((2.765003012084519 / (35)) + (1.7718371153659143 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.07119930643531491 = (60.904882730606396 - 65.24126813954976) / 60.904882730606396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | af88ee90bacedb49b90f7dc62a3092b287fc6ed0 |
| Compare | 789b420913a93f243adb4a6b14b162dc99c01242 |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector4
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [TransformByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark.html>) | 97.97 ns | 110.31 ns | 1.13 | 0.15 | False | |||||
| [TransformVector2ByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark.html>) | 82.53 ns | 97.99 ns | 1.19 | 0.16 | False | |||||
| [DistanceSquaredJitOptimizeCanaryBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark.html>) | 41.30 ns | 43.97 ns | 1.06 | 0.21 | False | |||||
| [TransformVector3ByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark.html>) | 100.51 ns | 108.40 ns | 1.08 | 0.16 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector4*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 110.31150246736269 > 102.30289451780108.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -17.385997159763267 (T) = (0 -110.71597352728367) / Math.Sqrt((10.569591849417902 / (35)) + (5.477583578397243 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.12362112676294743 = (98.53496956420403 - 110.71597352728367) / 98.53496956420403 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.98974222652413 > 88.33612609499221.
IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -19.410575349532436 (T) = (0 -99.00418670103376) / Math.Sqrt((1.3514518621085845 / (35)) + (17.568301778158975 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.18678408061367424 = (83.42224025270012 - 99.00418670103376) / 83.42224025270012 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.966419462708515 > 43.29864176654762.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -4.656290133237657 (T) = (0 -44.593587768686135) / Math.Sqrt((5.035542923585861 / (38)) + (4.150414153027851 / (26))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (26) - 2, .025) and -0.05981313881350048 = (42.07683990274954 - 44.593587768686135) / 42.07683990274954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 108.39647505785852 > 103.58246817439989.
IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -18.381704917240988 (T) = (0 -110.80551630745937) / Math.Sqrt((7.3134360640430085 / (34)) + (8.656663369022134 / (30))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (30) - 2, .025) and -0.13344143989538354 = (97.76024804394544 - 110.80551630745937) / 97.76024804394544 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | af88ee90bacedb49b90f7dc62a3092b287fc6ed0 |
| Compare | 789b420913a93f243adb4a6b14b162dc99c01242 |
| Diff | Diff |
Regressions in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.ContainsFalse(Int32).LinkedList(Size%3a%20512).html>) | 24.12 ms | 25.88 ms | 1.07 | 0.05 | False | |||||
| [ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.ContainsFalse(Int32).ImmutableList(Size%3a%20512).html>) | 33.03 ms | 35.29 ms | 1.07 | 0.03 | False | |||||
| [SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.ContainsFalse(Int32).SortedSet(Size%3a%20512).html>) | 484.20 μs | 530.88 μs | 1.10 | 0.08 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse<Int32>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Collections.ContainsFalse<Int32>.LinkedList(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.880245548148153 > 25.284234717807696.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -21.491963924699846 (T) = (0 -26159701.30816579) / Math.Sqrt((140080333978.49927 / (35)) + (147613668211.14514 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.0841348680561399 = (24129563.65389325 - 26159701.30816579) / 24129563.65389325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<Int32>.ImmutableList(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.28758488571429 > 34.5770359739011.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -24.61463717176828 (T) = (0 -35366075.84638671) / Math.Sqrt((43674106135.44429 / (35)) + (271748358231.89142 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.07602874636735006 = (32867222.149763025 - 35366075.84638671) / 32867222.149763025 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 530.8812597701149 > 509.8070989746094.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -12.916082227670486 (T) = (0 -527995.6843890157) / Math.Sqrt((80298684.71424282 / (35)) + (175037473.07062998 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.07497432338634782 = (491170.5078924504 - 527995.6843890157) / 491170.5078924504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | af88ee90bacedb49b90f7dc62a3092b287fc6ed0 |
| Compare | 789b420913a93f243adb4a6b14b162dc99c01242 |
| Diff | Diff |
Regressions in System.Collections.TryGetValueTrue<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.TryGetValueTrue(Int32%2c%20Int32).SortedDictionary(Size%3a%20512).html>) | 558.93 μs | 610.59 μs | 1.09 | 0.10 | False | |||||
| [ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.TryGetValueTrue(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) | 533.56 μs | 592.32 μs | 1.11 | 0.04 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue<Int32, Int32>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 610.5867367788461 > 583.5477222656251.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -18.56608005946948 (T) = (0 -603826.2070683208) / Math.Sqrt((157716428.3132816 / (35)) + (68557317.93377624 / (29))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (29) - 2, .025) and -0.08765683029237484 = (555162.4283056314 - 603826.2070683208) / 555162.4283056314 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueTrue<Int32, Int32>.ImmutableSortedDictionary(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 592.3173149691359 > 559.0295031291446.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -16.688440306013263 (T) = (0 -569056.7574460427) / Math.Sqrt((48019206.20968566 / (35)) + (92003307.45796165 / (31))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (31) - 2, .025) and -0.06506898925179325 = (534290.9832026964 - 569056.7574460427) / 534290.9832026964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | af88ee90bacedb49b90f7dc62a3092b287fc6ed0 |
| Compare | 789b420913a93f243adb4a6b14b162dc99c01242 |
| Diff | Diff |
Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) | 589.01 μs | 622.98 μs | 1.06 | 0.03 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse<Int32, Int32>*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
### Edge Detector Info
[Collection Data]()
#### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 622.9836670833333 > 621.1781059945913.
IsChangePoint: Marked as a change because one of 8/1/2022 7:13:37 AM, 9/28/2022 6:40:13 PM, 10/4/2022 5:30:44 AM falls between 9/25/2022 5:06:47 PM and 10/4/2022 5:30:44 AM.
IsRegressionStdDev: Marked as regression because -11.94488663698173 (T) = (0 -632724.2995194475) / Math.Sqrt((157033385.60357687 / (35)) + (100465945.6511378 / (28))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (28) - 2, .025) and -0.05668625794475149 = (598781.6106836598 - 632724.2995194475) / 598781.6106836598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos
Run Information
Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>
Test Report
Repro
Run Information
Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>
Test Report
Repro
Run Information
Regressions in System.Collections.ContainsTrue<Int32>
Test Report
Repro