performanceautofiler[bot]
commented
1 year ago Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c2469455eab98fdcea71997d3cc5004a992fc88f |
| Compare | 58614a0f6da787c9d6d914b99564f066d13f69ed |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Plane
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Plane.TransformByMatrix4x4Benchmark.html>) | 343.60 ns | 468.23 ns | 1.36 | 0.15 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Plane*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Numerics.Tests.Perf_Plane.TransformByMatrix4x4Benchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 468.2258053210501 > 362.23444256784654.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -32.55710281692139 (T) = (0 -468.4963327178757) / Math.Sqrt((119.76958817088172 / (34)) + (140.35120047843324 / (13))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (13) - 2, .025) and -0.3567833765394711 = (345.2992871366053 - 468.4963327178757) / 345.2992871366053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | c2469455eab98fdcea71997d3cc5004a992fc88f |
| Compare | 58614a0f6da787c9d6d914b99564f066d13f69ed |
| Diff | Diff |
Regressions in System.Numerics.Tests.Perf_Vector2
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| [TransformNormalByMatrix3x2Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark.html>) | 23.06 ns | 43.48 ns | 1.89 | 0.25 | False | |||||
| [TransformByMatrix3x2Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark.html>) | 25.59 ns | 50.48 ns | 1.97 | 0.26 | False | |||||
| [TransformNormalByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark.html>) | 24.24 ns | 107.76 ns | 4.45 | 0.29 | False | |||||
| [TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_LLVM=false_MonoAOT=false_MonoInterpreter=true/System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark.html>) | 26.79 ns | 112.35 ns | 4.19 | 0.27 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
### Payloads
[Baseline]()
[Compare]()
### Histogram
#### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 43.476436815181685 > 23.340865596565624.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -20.140748939390733 (T) = (0 -45.4310018173927) / Math.Sqrt((2.261593482174351 / (33)) + (15.731992988000682 / (13))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (13) - 2, .025) and -1.0052489587626552 = (22.65604059728625 - 45.4310018173927) / 22.65604059728625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.482460897405296 > 27.071664006549383.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -44.83973328146566 (T) = (0 -48.247193038979454) / Math.Sqrt((3.0618053709476287 / (33)) + (1.9745941609135151 / (13))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (13) - 2, .025) and -0.8508607244619394 = (26.067435761815798 - 48.247193038979454) / 26.067435761815798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.75555081975037 > 23.577778331392476.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -127.20368996730274 (T) = (0 -108.44295246767471) / Math.Sqrt((1.6061904074377689 / (34)) + (5.3069369581862 / (13))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (13) - 2, .025) and -3.799344364068183 = (22.595368083933995 - 108.44295246767471) / 22.595368083933995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark
```log
```
### Description of detection logic
```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 112.34851599855237 > 27.105507272478665.
IsChangePoint: Marked as a change because one of 1/6/2023 6:58:01 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsRegressionStdDev: Marked as regression because -119.75450732199337 (T) = (0 -111.03538506381506) / Math.Sqrt((1.3654393959918436 / (34)) + (6.006155511888729 / (13))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (13) - 2, .025) and -3.242454216054943 = (26.17244156545473 - 111.03538506381506) / 26.17244156545473 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```
### Docs
[Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md)
[Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
kotlarmilos
Run Information
Regressions in System.Numerics.Tests.Perf_Matrix4x4
Test Report
Repro