search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 52 Improvements on 10/20/2022 5:12:08 PM #9235

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 31d4e3aa7e4402539ded176c0de6796ad20cfc71
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadStructFieldByFieldUsingBitConverterBE - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE.html>) 1.37 μs 1.18 μs 0.86 0.05 False
[ReadStructFieldByFieldUsingBitConverterLE - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE.html>) 931.56 ns 822.44 ns 0.88 0.03 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterBE ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 1.1819675848561622 < 1.3045910340291111. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 45.53727856614312 (T) = (0 -1189.2831363808734) / Math.Sqrt((824.1148754870428 / (299)) + (340.0603631699892 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.1618305733826186 = (1418.9054129312367 - 1189.2831363808734) / 1418.9054129312367 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldUsingBitConverterLE ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 822.4362549965485 < 880.0075004411677. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 19.634160182146417 (T) = (0 -797.3984909892757) / Math.Sqrt((833.5099811166843 / (299)) + (127.11848366230524 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.07464441438334392 = (861.7211625279056 - 797.3984909892757) / 861.7211625279056 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 31d4e3aa7e4402539ded176c0de6796ad20cfc71
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SliceArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).SliceArray.html>) 232.55 ns 199.27 ns 0.86 0.09 False
[SliceSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Char).SliceSingleSegment.html>) 240.98 ns 204.32 ns 0.85 0.10 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 199.27026558804664 < 221.31949332510928. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 22.976813760199075 (T) = (0 -200.52010151655224) / Math.Sqrt((30.175892200916966 / (299)) + (29.036190680085966 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.13688682271844008 = (232.3219095647519 - 200.52010151655224) / 232.3219095647519 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Char>.SliceSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 204.32320062880507 < 229.0234244825851. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 63.58217874938674 (T) = (0 -202.05304809551205) / Math.Sqrt((34.90771424109153 / (299)) + (3.411267434338034 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.15583994996609218 = (239.35395673770168 - 202.05304809551205) / 239.35395673770168 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 31d4e3aa7e4402539ded176c0de6796ad20cfc71
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SliceArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceArray.html>) 240.05 ns 199.18 ns 0.83 0.12 False
[SliceSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceSingleSegment.html>) 237.01 ns 200.86 ns 0.85 0.09 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 199.17614763857648 < 224.69768975348452. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 12.647992246383245 (T) = (0 -201.42678194774217) / Math.Sqrt((43.306041286646106 / (299)) + (87.03450900838881 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.13279457378672366 = (232.27112730058522 - 201.42678194774217) / 232.27112730058522 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 200.8594044856006 < 225.61996170295092. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 34.69537961795076 (T) = (0 -202.6287793376648) / Math.Sqrt((29.11555306801057 / (299)) + (16.94621979159732 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.15190317937286404 = (238.92175328263627 - 202.6287793376648) / 238.92175328263627 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.SubtractBenchmark.html>) 244.60 ns 213.60 ns 0.87 0.10 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.AddOperatorBenchmark.html>) 203.92 ns 177.44 ns 0.87 0.13 False
[NormalizeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmark.html>) 174.18 ns 150.76 ns 0.87 0.12 False
[InverseBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.InverseBenchmark.html>) 184.16 ns 151.59 ns 0.82 0.12 False
[NegationOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark.html>) 142.06 ns 111.70 ns 0.79 0.12 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.AddBenchmark.html>) 242.97 ns 213.62 ns 0.88 0.08 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.SubtractionOperatorBenchmark.html>) 211.36 ns 175.70 ns 0.83 0.14 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark.html>) 159.76 ns 129.79 ns 0.81 0.09 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.NegateBenchmark.html>) 152.44 ns 122.13 ns 0.80 0.13 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarOperatorBenchmark.html>) 143.17 ns 116.46 ns 0.81 0.12 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_Quaternion.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 213.60331685965275 < 232.99935935521853. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 46.414683257652314 (T) = (0 -215.11441281122245) / Math.Sqrt((63.751040217397055 / (299)) + (2.700027478862019 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11766126728932713 = (243.80026041740194 - 215.11441281122245) / 243.80026041740194 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 177.43688094044197 < 195.05311345028608. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 15.463723545077956 (T) = (0 -180.2564538106647) / Math.Sqrt((26.821965644814213 / (299)) + (35.95752394142281 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.11299038851299913 = (203.2181517272165 - 180.2564538106647) / 203.2181517272165 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.NormalizeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 150.76026420606198 < 165.44533687657156. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 8.570871226305496 (T) = (0 -149.02841261632454) / Math.Sqrt((66.22654273447489 / (299)) + (17.15390949747329 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.06296401262155572 = (159.04235762946837 - 149.02841261632454) / 159.04235762946837 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.InverseBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 151.59467415781208 < 176.05688479365406. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 11.515238317401282 (T) = (0 -151.0471822744269) / Math.Sqrt((126.45198549044382 / (299)) + (11.705361138873727 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.07439349873261183 = (163.18725297154384 - 151.0471822744269) / 163.18725297154384 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 111.69989312668841 < 135.52327609994475. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 14.567179271053083 (T) = (0 -111.00963086653508) / Math.Sqrt((56.73701211539339 / (299)) + (10.439413776529829 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.10748390709752524 = (124.37829608823098 - 111.00963086653508) / 124.37829608823098 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 213.616612085448 < 230.59356510672905. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.71237340630805 (T) = (0 -215.27955444652986) / Math.Sqrt((31.36820155479746 / (299)) + (5.986289192972488 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11068632489494006 = (242.0738154297443 - 215.27955444652986) / 242.0738154297443 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 175.70080590700587 < 195.54040510179692. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 22.052721153972076 (T) = (0 -179.34420988568985) / Math.Sqrt((24.1036757938202 / (299)) + (18.6216168294415 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.11765671597960312 = (203.25899582814074 - 179.34420988568985) / 203.25899582814074 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 129.79217405743927 < 148.99593793125652. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 4:51:05 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 17.79389768439505 (T) = (0 -128.16050896479535) / Math.Sqrt((45.85966913984082 / (299)) + (4.7722961389493195 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.08704079587389162 = (140.37922876025067 - 128.16050896479535) / 140.37922876025067 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 122.13219079132068 < 145.30191027624062. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 23.41746614206001 (T) = (0 -123.09637479072029) / Math.Sqrt((50.191270097234906 / (299)) + (3.2179066130793466 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10208388564372788 = (137.09117457923082 - 123.09637479072029) / 137.09117457923082 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 116.45683518771386 < 137.54970461505533. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 18.877073562965876 (T) = (0 -114.91415371881283) / Math.Sqrt((61.869232477647614 / (299)) + (3.0510273719871415 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.09386086051255776 = (126.81733821123105 - 114.91415371881283) / 126.81733821123105 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark.html>) 119.70 ns 94.96 ns 0.79 0.10 False
[SubtractOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark.html>) 140.58 ns 102.28 ns 0.73 0.12 False
[CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark.html>) 114.65 ns 73.37 ns 0.64 0.10 False
[CreateScaleFromScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark.html>) 58.88 ns 51.02 ns 0.87 0.11 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark.html>) 135.78 ns 102.33 ns 0.75 0.07 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark.html>) 153.80 ns 120.98 ns 0.79 0.09 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark.html>) 132.70 ns 115.65 ns 0.87 0.09 False
[CreateRotationWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark.html>) 252.75 ns 199.00 ns 0.79 0.20 False
[CreateScaleFromScalarXYWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark.html>) 116.27 ns 75.23 ns 0.65 0.09 False
[CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark.html>) 113.05 ns 99.40 ns 0.88 0.08 False
[MultiplyByMatrixOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark.html>) 216.18 ns 154.08 ns 0.71 0.09 False
[MultiplyByMatrixBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixBenchmark.html>) 232.36 ns 176.64 ns 0.76 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 94.96105966190129 < 113.42407903893081. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 18.871459414274025 (T) = (0 -96.36753406567706) / Math.Sqrt((28.140110699576333 / (299)) + (5.597263042953419 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11541868461280493 = (108.94140808693827 - 96.36753406567706) / 108.94140808693827 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 102.28089108168803 < 128.88957728083096. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 26.78167152821755 (T) = (0 -103.91194583946229) / Math.Sqrt((28.06466682613825 / (299)) + (7.722883995066697 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.16741995885849223 = (124.80715451332938 - 103.91194583946229) / 124.80715451332938 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 73.36730974068072 < 108.75551596643118. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 25.601336119999317 (T) = (0 -74.74210958885779) / Math.Sqrt((50.55413486477266 / (299)) + (10.576126301620038 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2335273513366398 = (97.51438582863902 - 74.74210958885779) / 97.51438582863902 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 51.022798682623616 < 56.30588974889965. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.378194650938944 (T) = (0 -52.58547509595238) / Math.Sqrt((18.24506413366361 / (299)) + (2.278598465870632 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.24375788908199003 = (69.5352379043245 - 52.58547509595238) / 69.5352379043245 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 102.32631234188652 < 129.10514994740134. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 23.26508725712533 (T) = (0 -102.92656122695907) / Math.Sqrt((26.168930417783763 / (299)) + (12.820496832701997 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.17175407043872604 = (124.27053071239334 - 102.92656122695907) / 124.27053071239334 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 120.97702864642777 < 146.95384098835373. IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 32.871562893623434 (T) = (0 -122.74966606519448) / Math.Sqrt((40.99237849531452 / (299)) + (4.63942485246609 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.14637360346006245 = (143.79787991883114 - 122.74966606519448) / 143.79787991883114 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 115.64791446183114 < 124.41311850077761. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 16.349681525723796 (T) = (0 -106.11521297732331) / Math.Sqrt((27.3814182264441 / (299)) + (8.816514797040478 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1099329120670726 = (119.2215894913753 - 106.11521297732331) / 119.2215894913753 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 198.9951167865291 < 239.3060126783642. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 28.338433605258192 (T) = (0 -206.5942473826691) / Math.Sqrt((168.53091834039853 / (299)) + (110.32186112792839 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2670191525142684 = (281.85490533801527 - 206.5942473826691) / 281.85490533801527 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 75.22885702881672 < 110.15088124642689. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 39.28523006315602 (T) = (0 -75.41707291325143) / Math.Sqrt((51.506464939871634 / (299)) + (3.128368785685451 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.24006689625224817 = (99.24172606946331 - 75.41707291325143) / 99.24172606946331 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 99.40071040266275 < 109.49012338461351. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 16.10696244512753 (T) = (0 -101.22981524504996) / Math.Sqrt((12.064582917287934 / (299)) + (18.826249179764137 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1493191255318696 = (118.9985790010169 - 101.22981524504996) / 118.9985790010169 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 154.0771641801683 < 204.80634752472966. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 15.29263166721399 (T) = (0 -155.4633514117558) / Math.Sqrt((267.1351021686131 / (299)) + (14.967820727700326 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.11741122709881013 = (176.1447190187187 - 155.4633514117558) / 176.1447190187187 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByMatrixBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 176.63759545648242 < 221.14328318660378. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 14.810390084358115 (T) = (0 -175.90849871952528) / Math.Sqrt((249.00959096617044 / (299)) + (12.605001873106394 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.0955448972168322 = (194.4911341405713 - 175.90849871952528) / 194.4911341405713 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 31d4e3aa7e4402539ded176c0de6796ad20cfc71
Compare 4fc3eb443a3f8f8917038fbdba9049caf288020c
Diff Diff

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Slice_Start_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment%3a%20Single).html>) 228.02 ns 189.94 ns 0.83 0.08 False
[Slice_Repeat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat(Segment%3a%20Single).html>) 1.07 μs 906.61 ns 0.85 0.09 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySequence*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 189.94444005631507 < 218.07107928493443. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 4:51:05 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 7.003189989641857 (T) = (0 -195.6417954190119) / Math.Sqrt((47.18259767676372 / (299)) + (100.3750828966475 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.08565864991253429 = (213.97019329848405 - 195.6417954190119) / 213.97019329848405 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 906.6139819414609 < 1.0133660230316548. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 26.403636121186654 (T) = (0 -904.5975773671491) / Math.Sqrt((1910.4033202521714 / (299)) + (179.55404433190392 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.10728142656076513 = (1013.3065495457879 - 904.5975773671491) / 1013.3065495457879 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Improvements in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LerpBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark.html>) 381.91 ns 312.40 ns 0.82 0.06 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark.html>) 290.31 ns 207.81 ns 0.72 0.08 False
[CreateBillboardBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark.html>) 681.11 ns 563.22 ns 0.83 0.03 False
[CreateRotationXWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark.html>) 178.40 ns 126.04 ns 0.71 0.16 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark.html>) 237.39 ns 146.70 ns 0.62 0.15 False
[CreateScaleFromScalarXYZBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark.html>) 148.47 ns 113.63 ns 0.77 0.18 False
[MultiplyByMatrixBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark.html>) 620.36 ns 530.34 ns 0.85 0.04 False
[CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark.html>) 533.52 ns 391.69 ns 0.73 0.08 False
[CreateTranslationFromScalarXYZ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ.html>) 83.67 ns 70.35 ns 0.84 0.18 False
[CreateFromQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark.html>) 341.85 ns 242.68 ns 0.71 0.09 False
[CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark.html>) 149.21 ns 103.76 ns 0.70 0.15 False
[CreateRotationZWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark.html>) 166.27 ns 124.22 ns 0.75 0.15 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark.html>) 267.60 ns 190.77 ns 0.71 0.07 False
[CreateConstrainedBillboardBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark.html>) 857.76 ns 724.83 ns 0.85 0.06 False
[CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark.html>) 229.77 ns 154.76 ns 0.67 0.13 False
[CreateScaleFromScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark.html>) 79.08 ns 64.24 ns 0.81 0.22 False
[CreateShadowBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark.html>) 658.35 ns 576.09 ns 0.88 0.05 False
[CreateScaleFromScalarXYZWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark.html>) 152.21 ns 106.25 ns 0.70 0.14 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark.html>) 242.53 ns 167.25 ns 0.69 0.12 False
[NegationOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark.html>) 229.58 ns 152.23 ns 0.66 0.10 False
[CreateFromAxisAngleBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark.html>) 301.67 ns 213.95 ns 0.71 0.10 False
[CreateScaleFromVectorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark.html>) 142.48 ns 111.74 ns 0.78 0.16 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_Matrix4x4.LerpBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 312.40049403564245 < 361.49697044279594. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 21.326279085733045 (T) = (0 -290.8861577338727) / Math.Sqrt((476.88903284960077 / (299)) + (39.694985361091774 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.12689685600470252 = (333.16356691007337 - 290.8861577338727) / 333.16356691007337 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 207.80723687854478 < 276.0016439263795. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 32.918105729123084 (T) = (0 -208.93560577421522) / Math.Sqrt((308.97351933622855 / (299)) + (9.090726763681408 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.16478200277049568 = (250.15697275115483 - 208.93560577421522) / 250.15697275115483 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 563.2239650352328 < 661.0474451306213. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 65.06003035656838 (T) = (0 -572.7698065938777) / Math.Sqrt((1748.621793845211 / (299)) + (93.8382381446843 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2769249256902253 = (792.1304812513779 - 572.7698065938777) / 792.1304812513779 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 126.04405353311633 < 168.3566358721209. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 31.108392467999543 (T) = (0 -126.19473249124809) / Math.Sqrt((96.61826154475112 / (299)) + (12.540367750784066 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2024851750622366 = (158.23496760840288 - 126.19473249124809) / 158.23496760840288 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 146.7002789097244 < 220.7778136072152. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 31.370915058203835 (T) = (0 -150.83992255068608) / Math.Sqrt((347.8801445811022 / (299)) + (10.164061002452662 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.21631126323437047 = (192.47427642410582 - 150.83992255068608) / 192.47427642410582 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 113.62573693075858 < 142.64339043808627. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.59612191218295 (T) = (0 -109.26069342338015) / Math.Sqrt((38.17101323023518 / (299)) + (7.686300637740152 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.21596640270604942 = (139.35715739795782 - 109.26069342338015) / 139.35715739795782 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByMatrixBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 530.3361630654354 < 590.2877086064517. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 13.895694918714664 (T) = (0 -534.3020127258876) / Math.Sqrt((411.1676505751475 / (299)) + (110.20145206054984 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.06955763495296816 = (574.2451470370008 - 534.3020127258876) / 574.2451470370008 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 391.69184638679184 < 509.6072206799743. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 30.349409213771594 (T) = (0 -392.2390929748079) / Math.Sqrt((788.7087498972927 / (299)) + (59.543025462277036 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.16326422407541483 = (468.77294393368965 - 392.2390929748079) / 468.77294393368965 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 70.35243587750838 < 78.32010708279714. IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 44.167369200604085 (T) = (0 -69.36626884776815) / Math.Sqrt((25.21505066770482 / (299)) + (2.581930800890059 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.23632534699518506 = (90.8322262298927 - 69.36626884776815) / 90.8322262298927 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 242.67618532122026 < 317.33406620480014. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 48.27894079083577 (T) = (0 -242.54974674372488) / Math.Sqrt((260.6519379936867 / (299)) + (8.154716761803066 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.18959297913517348 = (299.2937382068676 - 242.54974674372488) / 299.2937382068676 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 103.75641135017561 < 138.9061286401503. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 4:51:05 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 37.1808281147489 (T) = (0 -105.74682629929202) / Math.Sqrt((54.398039157382094 / (299)) + (8.2928936403922 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2273349309527705 = (136.859851099116 - 105.74682629929202) / 136.859851099116 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 124.21575158313114 < 157.68605932803112. IsChangePoint: Marked as a change because one of 10/6/2022 10:14:50 PM, 10/19/2022 1:55:59 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.04908923386341 (T) = (0 -125.15343701712985) / Math.Sqrt((34.17614600877345 / (299)) + (9.726934549480411 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2012087510237114 = (156.6785279352064 - 125.15343701712985) / 156.6785279352064 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 190.76585503397504 < 254.53390111033153. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 35.36019393734433 (T) = (0 -186.9268414091956) / Math.Sqrt((310.31948910254135 / (299)) + (4.386944997618465 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.17809052277196044 = (227.42996228687184 - 186.9268414091956) / 227.42996228687184 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 724.8253516411341 < 809.2476260864518. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 60.8192936807032 (T) = (0 -720.0943629158403) / Math.Sqrt((2654.7753826604944 / (299)) + (108.11520000288097 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2503614087080464 = (960.5887040511138 - 720.0943629158403) / 960.5887040511138 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 154.76124081117615 < 218.53837097049802. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 36.78261749263206 (T) = (0 -157.1307470023504) / Math.Sqrt((201.61936856976072 / (299)) + (8.530064652050607 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.20246493500212234 = (197.0204871214892 - 157.1307470023504) / 197.0204871214892 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 64.23561895890113 < 74.44098665749372. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 36.965431931946675 (T) = (0 -66.0177170909538) / Math.Sqrt((24.54044423693551 / (299)) + (4.033681660698943 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.24909538452308005 = (87.91758064907371 - 66.0177170909538) / 87.91758064907371 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 576.0881235328638 < 626.6928239493883. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.66327161799967 (T) = (0 -581.4483836261328) / Math.Sqrt((208.54735047093965 / (299)) + (97.59421628034227 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.15144269917668768 = (685.219941024587 - 581.4483836261328) / 685.219941024587 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 106.25498968135255 < 143.95055670849024. IsChangePoint: Marked as a change because one of 10/7/2022 12:48:42 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 49.13281464240214 (T) = (0 -107.6902064303775) / Math.Sqrt((39.41420205955019 / (299)) + (4.580419967121328 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.2278039230493301 = (139.45966529075886 - 107.6902064303775) / 139.45966529075886 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 167.24830346387475 < 232.12520329181027. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 28.888859834129214 (T) = (0 -166.28655076689427) / Math.Sqrt((335.2318460050182 / (299)) + (11.402611604168374 / (16))) is greater than 1.9675720194858106 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (16) - 2, .975) and 0.1904559265272629 = (205.4076562546712 - 166.28655076689427) / 205.4076562546712 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 152.23393352630688 < 218.2062137566824. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 24.82729358129041 (T) = (0 -151.04093279222775) / Math.Sqrt((271.72928957820125 / (299)) + (27.846231645839893 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.2078083593773358 = (190.66211387122095 - 151.04093279222775) / 190.66211387122095 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 213.95087182838503 < 286.1832100607524. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 49.7683437915625 (T) = (0 -217.40690254909327) / Math.Sqrt((124.16779016194417 / (299)) + (32.35533535923824 / (17))) is greater than 1.9675476980854363 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (17) - 2, .975) and 0.25847219856223314 = (293.18779704221146 - 217.40690254909327) / 293.18779704221146 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 111.73533358469298 < 135.02561911449007. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsImprovementStdDev: Marked as improvement because 38.087212039504195 (T) = (0 -111.14870503959781) / Math.Sqrt((22.638437484861033 / (299)) + (8.473768295768524 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.21523722777990636 = (141.63350884389962 - 111.14870503959781) / 141.63350884389962 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

looks like https://github.com/dotnet/runtime/pull/77221