search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 11 Regressions on 2/2/2023 8:18:56 AM #12510

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2ba2396495c22429035d165e478672c442f81e22
Compare 6aa9f8b5a5d7ea4d79715f0b16f2a5b0ab6ac48d
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TransformNormalByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark.html>) 30.10 ns 96.18 ns 3.20 0.52 False
[TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark.html>) 30.96 ns 113.01 ns 3.65 0.52 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.18335303513464 > 31.19627333500133. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -62.93976548557434 (T) = (0 -98.67848897290943) / Math.Sqrt((1.8412829205651433 / (36)) + (26.364588166570922 / (23))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (23) - 2, .025) and -2.310779972918924 = (29.80520897796488 - 98.67848897290943) / 29.80520897796488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 113.01147892151991 > 33.46311659810486. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -75.24675126786067 (T) = (0 -111.90774577632655) / Math.Sqrt((4.846056498925285 / (37)) + (22.62681663119141 / (23))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (23) - 2, .025) and -2.447456350272149 = (32.46096089584784 - 111.90774577632655) / 32.46096089584784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2ba2396495c22429035d165e478672c442f81e22
Compare 6aa9f8b5a5d7ea4d79715f0b16f2a5b0ab6ac48d
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TransformVector2ByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark.html>) 42.62 ns 79.59 ns 1.87 0.48 False
[TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark.html>) 44.09 ns 123.31 ns 2.80 0.43 False
[TransformVector3ByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark.html>) 43.20 ns 114.81 ns 2.66 0.46 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 79.5946588894999 > 44.42503518670661. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -33.704867762909 (T) = (0 -83.53179220454872) / Math.Sqrt((9.627791820671794 / (37)) + (29.414212078080283 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -0.9678734415156286 = (42.44774610109769 - 83.53179220454872) / 42.44774610109769 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.31471322969485 > 45.55586698042127. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -52.80179888352099 (T) = (0 -112.4571764523862) / Math.Sqrt((4.738149492380817 / (37)) + (38.27226345687699 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -1.6059975477662745 = (43.1532165288408 - 112.4571764523862) / 43.1532165288408 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 114.80523797157527 > 45.49577553246151. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -47.172946416383326 (T) = (0 -113.81423720636883) / Math.Sqrt((5.0570713217745 / (37)) + (48.35085540425546 / (23))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (23) - 2, .025) and -1.6327756655286285 = (43.229751283619656 - 113.81423720636883) / 43.229751283619656 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2ba2396495c22429035d165e478672c442f81e22
Compare 6aa9f8b5a5d7ea4d79715f0b16f2a5b0ab6ac48d
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TransformNormalByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark.html>) 29.35 ns 75.57 ns 2.57 0.55 False
[TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark.html>) 31.07 ns 82.86 ns 2.67 0.55 False
[TransformByMatrix3x2Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark.html>) 5.22 ns 7.48 ns 1.43 0.66 False
[TransformNormalByMatrix3x2Benchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark.html>) 4.66 ns 7.46 ns 1.60 0.62 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 75.56750090974174 > 31.310160285995444. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -48.57078693459479 (T) = (0 -76.68630374376268) / Math.Sqrt((9.678813975839283 / (37)) + (16.362207441057585 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -1.5985328054253576 = (29.51138564949146 - 76.68630374376268) / 29.51138564949146 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.85522758081947 > 29.50777380912957. IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -53.23286355748858 (T) = (0 -82.70385527835937) / Math.Sqrt((4.08551470511349 / (37)) + (21.92553290917373 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -1.8680340741074353 = (28.836427023307856 - 82.70385527835937) / 28.836427023307856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.TransformByMatrix3x2Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.4846273661609946 > 5.780007102026343. IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 1/9/2023 6:56:09 PM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -6.625944619578451 (T) = (0 -7.72960061707566) / Math.Sqrt((1.5234226478275599 / (37)) + (0.5832070287606242 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -0.28097313794157835 = (6.034162925146512 - 7.72960061707566) / 6.034162925146512 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.45832205730421 > 4.888176074825902. IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -7.315892801330654 (T) = (0 -7.34350407869396) / Math.Sqrt((2.742991840028002 / (37)) + (0.6046570591640034 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -0.45768337116670904 = (5.037790938656537 - 7.34350407869396) / 5.037790938656537 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 2ba2396495c22429035d165e478672c442f81e22
Compare 6aa9f8b5a5d7ea4d79715f0b16f2a5b0ab6ac48d
Diff Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConjugateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark.html>) 15.42 ns 21.63 ns 1.40 0.58 False
[InverseBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.InverseBenchmark.html>) 16.25 ns 33.23 ns 2.04 0.61 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.62886130569762 > 16.965645188128914. IsChangePoint: Marked as a change because one of 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -10.68379698532365 (T) = (0 -22.639510503769465) / Math.Sqrt((4.11949076896121 / (37)) + (3.6661449512591178 / (23))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (23) - 2, .025) and -0.3254599588771334 = (17.0804937200431 - 22.639510503769465) / 17.0804937200431 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.InverseBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.22994786039278 > 18.147246864498616. IsChangePoint: Marked as a change because one of 2/2/2023 5:46:53 AM, 2/7/2023 2:48:42 AM falls between 1/29/2023 12:56:15 AM and 2/7/2023 2:48:42 AM. IsRegressionStdDev: Marked as regression because -20.091014900137647 (T) = (0 -30.36644033006925) / Math.Sqrt((1.9930951962990895 / (37)) + (8.529823331930816 / (24))) is less than -2.000995378087428 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (24) - 2, .025) and -0.7339107911818497 = (17.513265667705546 - 30.36644033006925) / 17.513265667705546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

@tannergooding looks like https://github.com/dotnet/runtime/pull/81335

tannergooding commented 1 year ago

Known/expected given Mono's lack of SIMD support for these types.

I'm working on adding that support in where its missing.