search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 17 Regressions on 10/20/2022 5:12:08 PM #9226

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.MinBenchmark.html>) 49.57 ns 53.91 ns 1.09 0.12 False
[CreateFromScalarXYBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.CreateFromScalarXYBenchmark.html>) 17.99 ns 20.02 ns 1.11 0.42 False
[UnitXBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector2.UnitXBenchmark.html>) 17.61 ns 19.81 ns 1.12 0.40 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_Vector2.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 53.90836728772564 > 52.192102232580815. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -11.678382988311105 (T) = (0 -53.47091628321746) / Math.Sqrt((1.6276581878212864 / (27)) + (0.4491629387999286 / (16))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (16) - 2, .025) and -0.06942813568641075 = (49.999541342623495 - 53.47091628321746) / 49.999541342623495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.CreateFromScalarXYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 20.020849260233668 > 18.891951501071794. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -3.040398735083701 (T) = (0 -21.509602517663307) / Math.Sqrt((1.5669638122269904 / (27)) + (19.676537093735295 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.19822907767835332 = (17.951160523778608 - 21.509602517663307) / 17.951160523778608 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector2.UnitXBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 19.806933864545737 > 19.530525750737283. IsChangePoint: Marked as a change because one of 10/19/2022 12:02:11 AM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -6.272620463497982 (T) = (0 -20.180629940261337) / Math.Sqrt((0.5892180548173731 / (21)) + (1.9826425334561377 / (21))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (21) - 2, .025) and -0.12205087740383387 = (17.985485637651873 - 20.180629940261337) / 17.985485637651873 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ZeroBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.ZeroBenchmark.html>) 11.77 ns 15.53 ns 1.32 0.55 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Numerics.Tests.Perf_Vector4.ZeroBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 15.532931749885313 > 12.857622486170971. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -5.653188089731575 (T) = (0 -15.099500684106511) / Math.Sqrt((1.282652093078265 / (25)) + (2.383397385950376 / (17))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (17) - 2, .025) and -0.19594445174288733 = (12.62558696777391 - 15.099500684106511) / 12.62558696777391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Memory.Constructors<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SpanFromMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).SpanFromMemory.html>) 47.30 ns 51.39 ns 1.09 0.13 False
[ReadOnlySpanFromMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).ReadOnlySpanFromMemory.html>) 46.84 ns 51.20 ns 1.09 0.20 False
[SpanImplicitCastFromArraySegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).SpanImplicitCastFromArraySegment.html>) 66.37 ns 73.55 ns 1.11 0.13 False
[MemoryMarshalCreateSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).MemoryMarshalCreateSpan.html>) 18.42 ns 20.26 ns 1.10 0.30 False
[ReadOnlySpanImplicitCastFromArraySegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).ReadOnlySpanImplicitCastFromArraySegment.html>) 66.06 ns 72.67 ns 1.10 0.11 False
[ArrayAsMemoryStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).ArrayAsMemoryStartLength.html>) 45.55 ns 49.52 ns 1.09 0.17 False
[SpanFromArrayStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).SpanFromArrayStartLength.html>) 37.07 ns 40.44 ns 1.09 0.16 False
[ReadOnlySpanImplicitCastFromArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(Byte).ReadOnlySpanImplicitCastFromArray.html>) 24.77 ns 27.18 ns 1.10 0.26 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors&lt;Byte&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Memory.Constructors<Byte>.SpanFromMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 51.38775783668422 > 49.42760397328483. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -9.52354453248163 (T) = (0 -51.465347997527886) / Math.Sqrt((1.0061018502330494 / (26)) + (2.1418870341406713 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.08119328317863099 = (47.60050658678108 - 51.465347997527886) / 47.60050658678108 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.ReadOnlySpanFromMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 51.20043132652609 > 49.40257396459526. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -6.320362738485598 (T) = (0 -51.5189810102206) / Math.Sqrt((2.0379010540198044 / (26)) + (4.701063724687125 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.07884914890283991 = (47.75364661742932 - 51.5189810102206) / 47.75364661742932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.SpanImplicitCastFromArraySegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 73.54678332937871 > 70.17464985148246. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -4.962440838408902 (T) = (0 -72.47431973995235) / Math.Sqrt((15.539182653020289 / (27)) + (2.602477482864762 / (17))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (17) - 2, .025) and -0.062074676684004806 = (68.23844060215303 - 72.47431973995235) / 68.23844060215303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.MemoryMarshalCreateSpan ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 20.259011195137013 > 19.174845432305027. IsChangePoint: Marked as a change because one of 8/5/2022 10:45:33 AM, 8/5/2022 8:01:14 PM, 10/19/2022 9:06:05 AM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -8.65787266565796 (T) = (0 -20.374959180901026) / Math.Sqrt((0.5369302027640743 / (23)) + (0.2924996860175278 / (20))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (20) - 2, .025) and -0.09027545769253914 = (18.687900417407015 - 20.374959180901026) / 18.687900417407015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.ReadOnlySpanImplicitCastFromArraySegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 72.67483398906354 > 70.13005727595181. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -5.375512931080072 (T) = (0 -73.0464339432401) / Math.Sqrt((0.9444313241100325 / (26)) + (19.56428823147438 / (16))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (16) - 2, .025) and -0.09000760444992811 = (67.01460948073198 - 73.0464339432401) / 67.01460948073198 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.ArrayAsMemoryStartLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 49.515789414937174 > 48.27428473941174. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -3.920300716111391 (T) = (0 -50.11318645899428) / Math.Sqrt((11.259566349496856 / (26)) + (1.3382156812060781 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.05928160391471186 = (47.30865359484629 - 50.11318645899428) / 47.30865359484629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.SpanFromArrayStartLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 40.43786112236727 > 38.77887782298202. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -8.312201208041092 (T) = (0 -40.480395872491435) / Math.Sqrt((1.135921546776454 / (26)) + (1.0447583693454163 / (17))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (17) - 2, .025) and -0.07133317168088636 = (37.78506718780959 - 40.480395872491435) / 37.78506718780959 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<Byte>.ReadOnlySpanImplicitCastFromArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 27.17814547113543 > 26.303073240900133. IsChangePoint: Marked as a change because one of 9/21/2022 6:35:49 PM, 9/22/2022 3:11:11 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -9.920151802634138 (T) = (0 -27.70146638132142) / Math.Sqrt((0.6759743401156498 / (26)) + (0.4956490226350351 / (16))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (16) - 2, .025) and -0.09347006386108787 = (25.333538884006025 - 27.70146638132142) / 25.333538884006025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare 4fc3eb443a3f8f8917038fbdba9049caf288020c
Diff Diff

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlySpanImplicitCastFromArraySegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(String).ReadOnlySpanImplicitCastFromArraySegment.html>) 69.90 ns 75.30 ns 1.08 0.10 False
[SpanImplicitCastFromArraySegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(String).SpanImplicitCastFromArraySegment.html>) 77.79 ns 82.04 ns 1.05 0.09 False
[ArrayAsSpanStartLength - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(String).ArrayAsSpanStartLength.html>) 45.95 ns 49.57 ns 1.08 0.12 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Constructors&lt;String&gt;*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 75.29605392724125 > 72.5324567604157. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -5.560196309323418 (T) = (0 -75.23044172555834) / Math.Sqrt((0.6985023789250367 / (27)) + (21.146497005899068 / (15))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (15) - 2, .025) and -0.0971603149316449 = (68.56832196874103 - 75.23044172555834) / 68.56832196874103 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<String>.SpanImplicitCastFromArraySegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 82.04311497604998 > 80.22390004332917. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -2.611193697774381 (T) = (0 -82.2373672686371) / Math.Sqrt((62.105647861504345 / (27)) + (1.3919266710674132 / (16))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (16) - 2, .025) and -0.05158945425591492 = (78.20292124061545 - 82.2373672686371) / 78.20292124061545 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<String>.ArrayAsSpanStartLength ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 49.571219090697106 > 48.28668880312213. IsChangePoint: Marked as a change because one of 8/23/2022 2:33:47 PM, 8/23/2022 11:46:46 PM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -13.901389778297153 (T) = (0 -49.36493471853746) / Math.Sqrt((0.5448232498460208 / (26)) + (0.34061856097848486 / (16))) is less than -2.0210753903043583 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (16) - 2, .025) and -0.06143449400462811 = (46.507754362015504 - 49.36493471853746) / 46.507754362015504 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 31d4e3aa7e4402539ded176c0de6796ad20cfc71
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StringAsSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.StringAsSpan.html>) 31.76 ns 36.13 ns 1.14 0.16 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Memory.ReadOnlySpan.StringAsSpan ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 36.1280266572016 > 34.09327409980055. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -5.806432675308599 (T) = (0 -35.841073830356414) / Math.Sqrt((2.8593084682986563 / (24)) + (1.1573957735077551 / (17))) is less than -2.0226909200346674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (24) + (17) - 2, .025) and -0.07538191781144198 = (33.328693031493586 - 35.841073830356414) / 33.328693031493586 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10d342cecc5cbfc57419f8a92af4787f29d76c08
Compare be7d6f557b80c211d10f29b57fa43dc55e8d66cf
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[op_Subtraction - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.op_Subtraction.html>) 55.95 ns 60.59 ns 1.08 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_DateTime*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_DateTime.op_Subtraction ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionWindowed: Marked as regression because 60.59268374247058 > 58.94667670107792. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 10/25/2022 8:17:52 AM falls between 10/16/2022 1:15:28 PM and 10/25/2022 8:17:52 AM. IsRegressionStdDev: Marked as regression because -4.306087708822842 (T) = (0 -60.244665036235105) / Math.Sqrt((7.812348964117513 / (27)) + (3.957796405793826 / (17))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (17) - 2, .025) and -0.05446233254491862 = (57.133065048265976 - 60.244665036235105) / 57.133065048265976 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

This might be caused by https://github.com/dotnet/runtime/pull/77221. @BrzVlad please take a look.