dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 1922 Regressions on 12/9/2022 5:51:37 PM #10711

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Slice_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment%3a%20Multiple).html>) 168.60 ns 197.92 ns 1.17 0.12 False
[Slice_Start_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment%3a%20Multiple).html>) 291.70 ns 350.13 ns 1.20 0.13 False
[Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment%3a%20Multiple).html>) 979.13 ns 1.13 μs 1.15 0.09 False
[Slice_Start_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment%3a%20Multiple).html>) 304.12 ns 361.95 ns 1.19 0.11 False
[Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment%3a%20Single).html>) 756.18 ns 896.40 ns 1.19 0.10 False
[Slice_Repeat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat(Segment%3a%20Multiple).html>) 1.24 μs 1.46 μs 1.18 0.06 False
[Slice_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment%3a%20Single).html>) 126.44 ns 157.65 ns 1.25 0.12 False
[Slice_StartPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition(Segment%3a%20Multiple).html>) 173.75 ns 200.26 ns 1.15 0.15 False
[Slice_Start_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment%3a%20Single).html>) 184.41 ns 232.57 ns 1.26 0.11 False
[Slice_StartPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition(Segment%3a%20Single).html>) 137.01 ns 153.56 ns 1.12 0.09 False
[Slice_StartPosition_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment%3a%20Single).html>) 226.88 ns 271.92 ns 1.20 0.19 False
[Slice_Repeat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat(Segment%3a%20Single).html>) 865.44 ns 1.02 μs 1.18 0.07 False
[Slice_Start_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment%3a%20Single).html>) 226.07 ns 266.70 ns 1.18 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 197.92235988743397 > 177.18567282127594. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.518877405406933 (T) = (0 -202.7045277235993) / Math.Sqrt((7.74880234240451 / (44)) + (55.15055086716372 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19041669801508174 = (170.28031281952934 - 202.7045277235993) / 170.28031281952934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 350.1305290205563 > 307.44407417885117. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.23696868966313 (T) = (0 -348.4570633390689) / Math.Sqrt((47.037409026319764 / (44)) + (25.147655277881256 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18104446377011776 = (295.04144342434654 - 348.4570633390689) / 295.04144342434654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1257044070730031 > 1018.4313616942279. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.76998596056479 (T) = (0 -1132.2783371880653) / Math.Sqrt((507.9592338522709 / (44)) + (783.3442909525414 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16201903219570601 = (974.406017299524 - 1132.2783371880653) / 974.406017299524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.9522386943119 > 320.58757056165683. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.51154440305134 (T) = (0 -359.21893417920535) / Math.Sqrt((25.091528603303146 / (44)) + (134.2726660427784 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17706109648765916 = (305.1829129780194 - 359.21893417920535) / 305.1829129780194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 896.3998395257353 > 791.1629829933098. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.129326084305122 (T) = (0 -907.5184643486697) / Math.Sqrt((749.3372694873966 / (44)) + (1165.9263934653638 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19372103424557513 = (760.2433385302758 - 907.5184643486697) / 760.2433385302758 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4632897004821581 > 1.280391183649244. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.53226989587857 (T) = (0 -1470.322082448716) / Math.Sqrt((472.1679724299574 / (44)) + (165.7637494100169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1955638150074219 = (1229.814807032771 - 1470.322082448716) / 1229.814807032771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 157.651999910994 > 133.27816282562117. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.04543256682334 (T) = (0 -155.43679116786527) / Math.Sqrt((2.533955291854354 / (44)) + (3.1965320233402688 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22586430331189639 = (126.79771386435218 - 155.43679116786527) / 126.79771386435218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.2626097402163 > 185.2991957050644. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 5:51:37 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.137246288330198 (T) = (0 -197.21144851002373) / Math.Sqrt((18.151887331455605 / (45)) + (32.62229025818346 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.11328053281699349 = (177.14443277922862 - 197.21144851002373) / 177.14443277922862 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 232.56969725060424 > 194.94879794989387. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.99315150844873 (T) = (0 -221.82693819480966) / Math.Sqrt((18.517376723056536 / (44)) + (89.19905872923223 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1893129310972052 = (186.51688079281402 - 221.82693819480966) / 186.51688079281402 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.56326433206314 > 144.38586512065675. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.616097757156435 (T) = (0 -155.99142699419) / Math.Sqrt((6.059356812812471 / (43)) + (7.492576447231934 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.12127432128517877 = (139.11977116839367 - 155.99142699419) / 139.11977116839367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 271.9177404578498 > 240.41243886641726. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.74384614382784 (T) = (0 -272.3166263147044) / Math.Sqrt((47.33209047975967 / (44)) + (3.647884122814358 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17168117569711702 = (232.41529518700662 - 272.3166263147044) / 232.41529518700662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0199767037539629 > 905.1480858010625. IsChangePoint: Marked as a change because one of 10/20/2022 5:12:08 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.91297953179902 (T) = (0 -1023.9921291190827) / Math.Sqrt((93.20411266984439 / (44)) + (145.52529598092596 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.18364363695718636 = (865.1186025478728 - 1023.9921291190827) / 865.1186025478728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.7036457369877 > 241.66848163482152. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.157850395118523 (T) = (0 -269.43697593752916) / Math.Sqrt((19.939290877412624 / (44)) + (27.358908916864227 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16400696603130746 = (231.47367996961137 - 269.43697593752916) / 231.47367996961137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 250.09 μs 449.92 μs 1.80 0.05 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 345.21 μs 478.99 μs 1.39 0.03 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) 642.46 μs 737.98 μs 1.15 0.03 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 250.32 μs 448.34 μs 1.79 0.04 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 247.65 μs 443.93 μs 1.79 0.02 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 449.92346938775506 > 263.4478261040707. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.93017220587247 (T) = (0 -449630.8336568549) / Math.Sqrt((243709322.8936953 / (44)) + (2590309.8868702245 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7714038731951897 = (253827.39671096476 - 449630.8336568549) / 253827.39671096476 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 478.9940641414141 > 356.33131760555193. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 10:11:20 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.60623622904413 (T) = (0 -483864.7697008473) / Math.Sqrt((264496248.83511817 / (44)) + (110852439.23642598 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.4560459655913539 = (332314.2133801607 - 483864.7697008473) / 332314.2133801607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 737.9793011363639 > 686.7939707161459. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.503492931924368 (T) = (0 -720843.0475647617) / Math.Sqrt((1839148387.745977 / (44)) + (30858818.99515411 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11817137814186991 = (644662.4029695952 - 720843.0475647617) / 644662.4029695952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 448.3428555952382 > 264.76808312872026. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -92.23129980004123 (T) = (0 -443571.72250951367) / Math.Sqrt((178256896.18906 / (44)) + (3623104.700536639 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7568847884266237 = (252476.2724519653 - 443571.72250951367) / 252476.2724519653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 443.9308226495727 > 264.5840930953581. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.75597907990853 (T) = (0 -441325.2574755801) / Math.Sqrt((162619496.96342447 / (43)) + (1821043.072159508 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.7641164635460395 = (250167.8696362682 - 441325.2574755801) / 250167.8696362682 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.Logging.LoggingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NoArguments_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel.html>) 319.26 ns 348.59 ns 1.09 0.10 False
[NoArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage.html>) 472.36 ns 522.53 ns 1.11 0.07 False
[TwoArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage.html>) 512.43 ns 563.69 ns 1.10 0.08 False
[NoArguments_DefineMessage_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel.html>) 140.48 ns 161.03 ns 1.15 0.08 False
[TwoArguments_DefineMessage_FilteredByLevel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel.html>) 145.17 ns 168.32 ns 1.16 0.08 False
[NoArguments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.LoggingOverhead.NoArguments.html>) 400.00 ns 434.27 ns 1.09 0.11 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.LoggingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_FilteredByLevel ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 348.5881123058543 > 335.38156870752107. IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.1607910206981 (T) = (0 -352.5131065186305) / Math.Sqrt((13.30585003629229 / (44)) + (25.91668408566175 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09353098063192851 = (322.3622492294828 - 352.5131065186305) / 322.3622492294828 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 522.5332378020602 > 506.4666610875171. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.011446160149454 (T) = (0 -525.4778776978176) / Math.Sqrt((63.344098077264796 / (44)) + (52.95330870334368 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09786378489163762 = (478.6366805511157 - 525.4778776978176) / 478.6366805511157 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 563.6862007581458 > 540.3269746104443. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.811779335481567 (T) = (0 -570.1023700530419) / Math.Sqrt((54.83376831473626 / (44)) + (62.573182090796976 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09775867527181288 = (519.3330582533365 - 570.1023700530419) / 519.3330582533365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.02999364055552 > 147.3135057768418. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.413695427562107 (T) = (0 -162.06333400801304) / Math.Sqrt((19.429516952108433 / (44)) + (1.5800817886322494 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14897560297530432 = (141.05028304199456 - 162.06333400801304) / 141.05028304199456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 168.31863864012817 > 150.8790716329598. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.30342650618993 (T) = (0 -165.20119155307358) / Math.Sqrt((14.405083055106086 / (44)) + (1.7735737196233647 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13953814104478068 = (144.9720598220693 - 165.20119155307358) / 144.9720598220693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.LoggingOverhead.NoArguments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 434.27061695926017 > 424.50093557310703. IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.960077127573772 (T) = (0 -439.63516077778837) / Math.Sqrt((67.19821516306908 / (43)) + (102.97955987782196 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.08335453785705842 = (405.809128420151 - 439.63516077778837) / 405.809128420151 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchF.BenchMrk

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchF.BenchMrk.Test.html>) 689.39 ms 761.40 ms 1.10 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.BenchMrk*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.BenchMrk.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 761.3965332666667 > 709.9966812125. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.046834750855712 (T) = (0 -749896670.4445299) / Math.Sqrt((138667300482094.39 / (44)) + (208179250509696.22 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1102643721557343 = (675421718.692549 - 749896670.4445299) / 675421718.692549 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonSourceGen.html>) 49.94 μs 55.65 μs 1.11 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonReflection.html>) 48.91 μs 55.90 μs 1.14 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromString(Location).JsonNet.html>) 67.48 μs 71.87 μs 1.07 0.00 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.647395702162605 > 52.21411763759991. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.69314972244291 (T) = (0 -55915.13313037266) / Math.Sqrt((392456.35763997317 / (44)) + (174261.27028897512 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13264219819184278 = (49366.98740311454 - 55915.13313037266) / 49366.98740311454 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.90454026284349 > 51.372648902821325. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.92760484578411 (T) = (0 -55797.55749211527) / Math.Sqrt((349416.9480269184 / (44)) + (334410.3090973014 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14210948165154133 = (48854.82380500818 - 55797.55749211527) / 48854.82380500818 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<Location>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.8736559715822 > 71.33542141069192. IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.866602966663784 (T) = (0 -72993.60983179173) / Math.Sqrt((1127726.8747592538 / (44)) + (322222.9853520707 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08682653809583307 = (67162.15262803546 - 72993.60983179173) / 67162.15262803546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).GreaterThanAnyBenchmark.html>) 72.21 ns 215.98 ns 2.99 0.10 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).GreaterThanAllBenchmark.html>) 52.28 ns 240.62 ns 4.60 0.12 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).MultiplyOperatorBenchmark.html>) 92.61 ns 111.24 ns 1.20 0.09 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).OnesComplementOperatorBenchmark.html>) 67.44 ns 144.35 ns 2.14 0.05 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).LessThanAllBenchmark.html>) 71.33 ns 298.40 ns 4.18 0.14 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).UnaryNegateOperatorBenchmark.html>) 91.80 ns 126.25 ns 1.38 0.09 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).LessThanAnyBenchmark.html>) 51.66 ns 158.57 ns 3.07 0.12 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).AndNotBenchmark.html>) 169.24 ns 259.72 ns 1.53 0.09 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).DivideBenchmark.html>) 104.81 ns 135.89 ns 1.30 0.10 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).SubtractionOperatorBenchmark.html>) 94.40 ns 107.09 ns 1.13 0.08 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).GreaterThanOrEqualAllBenchmark.html>) 55.89 ns 248.83 ns 4.45 0.11 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).NegateBenchmark.html>) 90.83 ns 122.99 ns 1.35 0.10 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).AddBenchmark.html>) 93.25 ns 124.87 ns 1.34 0.08 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).DivisionOperatorBenchmark.html>) 104.94 ns 121.51 ns 1.16 0.10 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).OneBenchmark.html>) 37.67 ns 76.52 ns 2.03 0.12 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).GreaterThanOrEqualAnyBenchmark.html>) 76.82 ns 222.25 ns 2.89 0.14 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).AddOperatorBenchmark.html>) 92.32 ns 108.39 ns 1.17 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).LessThanOrEqualAllBenchmark.html>) 74.62 ns 300.68 ns 4.03 0.11 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).EqualsBenchmark.html>) 81.18 ns 89.49 ns 1.10 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).EqualsAnyBenchmark.html>) 73.96 ns 215.73 ns 2.92 0.11 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).MultiplyBenchmark.html>) 95.97 ns 127.82 ns 1.33 0.08 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).SubtractBenchmark.html>) 94.86 ns 127.57 ns 1.34 0.12 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).LessThanOrEqualAnyBenchmark.html>) 53.19 ns 163.14 ns 3.07 0.08 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt64).OnesComplementBenchmark.html>) 66.83 ns 140.14 ns 2.10 0.09 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt64>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.9784833428843 > 75.8210981421205. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -109.87208128901064 (T) = (0 -217.95105973455404) / Math.Sqrt((10.640636689767112 / (44)) + (22.42860840624316 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.9798437922580066 = (73.141773505315 - 217.95105973455404) / 73.141773505315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.61693798009446 > 53.732898613982265. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -382.1456552960092 (T) = (0 -243.13624616953484) / Math.Sqrt((1.301393275019018 / (44)) + (3.314169112073214 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.6879446044468405 = (51.86414658972362 - 243.13624616953484) / 51.86414658972362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 111.23822416460548 > 97.13487663582164. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.721267107589897 (T) = (0 -110.16459753803318) / Math.Sqrt((8.215314783575081 / (44)) + (22.765388732565725 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17751789700597942 = (93.55662263660165 - 110.16459753803318) / 93.55662263660165 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.34781271060567 > 70.04455855744001. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.739481610356954 (T) = (0 -144.48570047348227) / Math.Sqrt((1.8739906950455174 / (44)) + (42.04571564655159 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1460075166672548 = (67.3276767911178 - 144.48570047348227) / 67.3276767911178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 298.3980361978729 > 75.66402880378406. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -219.1497313814326 (T) = (0 -296.72908545969216) / Math.Sqrt((8.8187176040032 / (44)) + (12.638687803182936 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.069423749794797 = (72.91673310616888 - 296.72908545969216) / 72.91673310616888 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 126.24708029226444 > 96.93260264005822. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.712140960949643 (T) = (0 -126.8981225016169) / Math.Sqrt((1.5487392430706082 / (44)) + (46.45553713194879 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3791574317973638 = (92.01133936989271 - 126.8981225016169) / 92.01133936989271 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.56649667179167 > 53.917743407930615. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -106.01124514040636 (T) = (0 -160.78104756340497) / Math.Sqrt((11.904706071178321 / (44)) + (11.739452828182504 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.0927240448018827 = (51.98687151983017 - 160.78104756340497) / 51.98687151983017 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 259.7242462992752 > 180.33839484269635. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -88.28549750430098 (T) = (0 -254.6062205948363) / Math.Sqrt((18.560067161434638 / (44)) + (6.595478128920557 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.4873502125065113 = (171.1810832808293 - 254.6062205948363) / 171.1810832808293 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.88969124310685 > 110.88754123322894. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.86093155023972 (T) = (0 -137.54726291758834) / Math.Sqrt((14.872829254184184 / (44)) + (20.80444207002731 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3112817899968544 = (104.8952742018314 - 137.54726291758834) / 104.8952742018314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.08503552159921 > 98.04337833871651. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.44771858292131 (T) = (0 -108.98854112698066) / Math.Sqrt((2.6895229916030163 / (44)) + (1.865665526215551 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16290194865257493 = (93.72117851660919 - 108.98854112698066) / 93.72117851660919 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 248.8305700149975 > 56.2338720980186. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -194.25398497878567 (T) = (0 -246.43906536437774) / Math.Sqrt((3.63762927391547 / (44)) + (13.482303889199784 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.564452183375505 = (53.99094030647311 - 246.43906536437774) / 53.99094030647311 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 122.98565797991643 > 96.48270748982125. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.88567534670187 (T) = (0 -125.60845547159138) / Math.Sqrt((10.164539260625613 / (44)) + (28.164882272479193 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.359758495756616 = (92.37556217782523 - 125.60845547159138) / 92.37556217782523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.86610268695888 > 98.36918609996837. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.50871878506693 (T) = (0 -126.06713029814595) / Math.Sqrt((3.019499042569447 / (44)) + (1.7715800216633846 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34920514700851896 = (93.438073948698 - 126.06713029814595) / 93.438073948698 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 121.51196467510835 > 108.43434799611963. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.423034172018943 (T) = (0 -121.26357086894681) / Math.Sqrt((17.60371302496945 / (44)) + (34.320559267136886 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16404281989021277 = (104.17449323761461 - 121.26357086894681) / 104.17449323761461 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.52444636048622 > 39.581838560327505. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.44271520763725 (T) = (0 -77.0321360297621) / Math.Sqrt((1.700968285489444 / (44)) + (3.422953296444888 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0239704236372216 = (38.059911908855746 - 77.0321360297621) / 38.059911908855746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 222.2540817902489 > 79.33166380393504. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -144.08648446515875 (T) = (0 -220.23937310495768) / Math.Sqrt((8.28373893431182 / (44)) + (11.944413183139277 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.8502759889663503 = (77.26949037830799 - 220.23937310495768) / 77.26949037830799 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.38875769644434 > 97.25138225259818. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.563234926428938 (T) = (0 -111.3908867174425) / Math.Sqrt((6.756755305247656 / (44)) + (23.869357689968634 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19166733106314265 = (93.4748178571493 - 111.3908867174425) / 93.4748178571493 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 300.68388141381564 > 79.10791129801058. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -192.32417741654882 (T) = (0 -302.3707912032505) / Math.Sqrt((4.485466583284366 / (44)) + (19.178497723781874 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.9577649083297315 = (76.39938152133902 - 302.3707912032505) / 76.39938152133902 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.48522575784615 > 86.18081273612307. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.043013989154831 (T) = (0 -89.89382701015602) / Math.Sqrt((3.582845685607474 / (44)) + (3.050040192245542 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09805552773814538 = (81.86637628001004 - 89.89382701015602) / 81.86637628001004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.72717707897112 > 75.94510583245231. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -93.96292395058505 (T) = (0 -218.8758534067053) / Math.Sqrt((0.7818973469551774 / (44)) + (36.07013796716679 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.0135741586853864 = (72.62998747712439 - 218.8758534067053) / 72.62998747712439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.82247724472481 > 99.4688572225333. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.295974101755974 (T) = (0 -126.00985810644202) / Math.Sqrt((17.39827179875237 / (43)) + (2.2677018907701503 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.33375354672261764 = (94.47761800977497 - 126.00985810644202) / 94.47761800977497 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.57488642595462 > 98.45337279598964. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.80025377982029 (T) = (0 -126.15568543749215) / Math.Sqrt((20.81900162161994 / (44)) + (2.1854848594000384 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3412380470232935 = (94.05913120156305 - 126.15568543749215) / 94.05913120156305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 163.14241671881516 > 55.6772689213307. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -248.39437312159663 (T) = (0 -162.3726135240341) / Math.Sqrt((1.096576869650072 / (44)) + (2.513199473297452 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.0407512772893277 = (53.398847428514735 - 162.3726135240341) / 53.398847428514735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.14159195646135 > 70.24652212233174. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -95.8134978480181 (T) = (0 -143.21661713595228) / Math.Sqrt((1.3238824716065694 / (44)) + (8.991230893331496 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1312955667463114 = (67.1969760414743 - 143.21661713595228) / 67.1969760414743 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).EqualsAnyBenchmark.html>) 72.06 ns 216.96 ns 3.01 0.09 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).GreaterThanOrEqualAllBenchmark.html>) 53.38 ns 267.01 ns 5.00 0.12 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).OnesComplementBenchmark.html>) 67.19 ns 144.10 ns 2.14 0.09 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).NegateBenchmark.html>) 93.75 ns 124.67 ns 1.33 0.09 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).MultiplyBenchmark.html>) 93.46 ns 129.38 ns 1.38 0.09 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).AddBenchmark.html>) 92.66 ns 135.71 ns 1.46 0.07 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).SubtractionOperatorBenchmark.html>) 93.40 ns 108.43 ns 1.16 0.11 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).MultiplyOperatorBenchmark.html>) 95.01 ns 108.67 ns 1.14 0.06 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).SubtractBenchmark.html>) 110.72 ns 127.41 ns 1.15 0.07 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).OneBenchmark.html>) 37.58 ns 76.73 ns 2.04 0.10 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).AddOperatorBenchmark.html>) 91.87 ns 108.42 ns 1.18 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).LessThanOrEqualAllBenchmark.html>) 75.32 ns 297.41 ns 3.95 0.14 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).LessThanOrEqualAnyBenchmark.html>) 53.71 ns 162.09 ns 3.02 0.09 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).DivisionOperatorBenchmark.html>) 105.69 ns 124.14 ns 1.17 0.07 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).GreaterThanAllBenchmark.html>) 51.11 ns 243.48 ns 4.76 0.10 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).LessThanAnyBenchmark.html>) 51.65 ns 158.64 ns 3.07 0.10 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).AndNotBenchmark.html>) 169.93 ns 254.28 ns 1.50 0.09 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).AbsBenchmark.html>) 72.43 ns 106.20 ns 1.47 0.08 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).GreaterThanAnyBenchmark.html>) 72.35 ns 215.18 ns 2.97 0.12 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).LessThanAllBenchmark.html>) 72.00 ns 296.48 ns 4.12 0.12 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).OnesComplementOperatorBenchmark.html>) 67.14 ns 141.61 ns 2.11 0.09 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).DivideBenchmark.html>) 105.54 ns 141.12 ns 1.34 0.09 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).UnaryNegateOperatorBenchmark.html>) 91.04 ns 123.98 ns 1.36 0.06 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int64).GreaterThanOrEqualAnyBenchmark.html>) 75.03 ns 220.75 ns 2.94 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int64>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.9640954658279 > 75.80089064173957. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -187.21021073749972 (T) = (0 -216.84448254015115) / Math.Sqrt((21.018847720069303 / (44)) + (1.5892784287241983 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.9374301987388673 = (73.8211524594693 - 216.84448254015115) / 73.8211524594693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 267.0123124222413 > 55.73371972965563. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.82808295843548 (T) = (0 -251.28650256775703) / Math.Sqrt((1.732430360009098 / (44)) + (245.1889502791153 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.684994401713357 = (53.63645738314194 - 251.28650256775703) / 53.63645738314194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.10063866269834 > 70.85007445395715. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -91.7361741995601 (T) = (0 -142.38030209152504) / Math.Sqrt((12.105421791833686 / (44)) + (5.358975567846892 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -1.0946986569753336 = (67.97173503568139 - 142.38030209152504) / 67.97173503568139 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.67020480475884 > 97.1679475831011. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.65179531258155 (T) = (0 -125.82662340752957) / Math.Sqrt((10.245941495647536 / (44)) + (29.196432934617995 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3619503391455756 = (92.38708621818571 - 125.82662340752957) / 92.38708621818571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.37732577601446 > 97.26094322828368. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.337388254611916 (T) = (0 -127.62482169489866) / Math.Sqrt((6.876074732229245 / (44)) + (15.396248133150584 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36431601996837293 = (93.54491175575085 - 127.62482169489866) / 93.54491175575085 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.71294223118832 > 97.24038834123994. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.87034829724055 (T) = (0 -127.34604590515218) / Math.Sqrt((2.475645981484009 / (44)) + (10.095671570712566 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36495442871793526 = (93.2969212933833 - 127.34604590515218) / 93.2969212933833 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.43263374855837 > 98.33523570772427. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.364628058012109 (T) = (0 -110.13973497939415) / Math.Sqrt((11.977961329069565 / (44)) + (20.312435241018672 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1670913912846764 = (94.37113134572759 - 110.13973497939415) / 94.37113134572759 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.66624856567901 > 97.72720881903908. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.014990886571372 (T) = (0 -109.93009467001717) / Math.Sqrt((1.6679637964879837 / (44)) + (13.229227003652753 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17432681376711748 = (93.6111595011383 - 109.93009467001717) / 93.6111595011383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.41196213321393 > 97.73355881181855. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.72223108953368 (T) = (0 -125.38611663699045) / Math.Sqrt((15.793197964494196 / (44)) + (1.1391047880345726 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34453448648805385 = (93.25615512064778 - 125.38611663699045) / 93.25615512064778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.72564334687536 > 39.7403495034536. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -102.0311527505627 (T) = (0 -76.62515934860912) / Math.Sqrt((1.809563441678793 / (44)) + (1.4825381503235562 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.9926583956937853 = (38.453735730218064 - 76.62515934860912) / 38.453735730218064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.42144159229359 > 98.18794386081866. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.68471622983179 (T) = (0 -108.91878784150964) / Math.Sqrt((23.202297287359023 / (43)) + (0.9322120616200743 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15349830560955272 = (94.42474888071273 - 108.91878784150964) / 94.42474888071273 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 297.40957936571033 > 79.14260194285391. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -129.61963096825278 (T) = (0 -302.609718027641) / Math.Sqrt((25.873873170142303 / (44)) + (36.65557573319497 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.9342306690177895 = (76.91712649456566 - 302.609718027641) / 76.91712649456566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 162.09485097937687 > 56.183121998433876. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -156.17482225505634 (T) = (0 -162.4723652516139) / Math.Sqrt((14.401347635421025 / (44)) + (2.3100290672127755 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.001840190918674 = (54.124255429430896 - 162.4723652516139) / 54.124255429430896 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.1436981164105 > 112.6739729379816. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.613066999473904 (T) = (0 -123.62410259019937) / Math.Sqrt((12.048658038297622 / (44)) + (1.0148430038673002 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15012185604175593 = (107.48783004234215 - 123.62410259019937) / 107.48783004234215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 243.47598265178272 > 54.531709743717336. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -198.97443741973242 (T) = (0 -244.60471514787562) / Math.Sqrt((0.720562606422428 / (44)) + (13.855158746175352 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.732483422629377 = (51.68633322163287 - 244.60471514787562) / 51.68633322163287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.64032515853594 > 53.656354165196916. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.91580933249855 (T) = (0 -160.81287514046178) / Math.Sqrt((1.7553931489132784 / (43)) + (27.38249224327048 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -2.1142859986757268 = (51.63715702695369 - 160.81287514046178) / 51.63715702695369 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 254.27870491907728 > 178.42781867064028. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -85.57456578618454 (T) = (0 -254.9328431917424) / Math.Sqrt((10.44621346003401 / (44)) + (10.460193051179102 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.49941950484418085 = (170.02102638262994 - 254.9328431917424) / 170.02102638262994 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 106.2047885205037 > 77.28520238011446. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.22585225470702 (T) = (0 -105.27948944554515) / Math.Sqrt((22.948488871421674 / (44)) + (1.2346928039618632 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4075276384718857 = (74.79745801641538 - 105.27948944554515) / 74.79745801641538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.18435636963696 > 77.34514751394838. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -148.82811527111917 (T) = (0 -217.15415098663362) / Math.Sqrt((17.859110371240135 / (44)) + (7.8045394262501 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.9375446389466338 = (73.92369399516677 - 217.15415098663362) / 73.92369399516677 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 296.4844722089479 > 76.1855192700081. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -134.45220123699724 (T) = (0 -297.22881358915913) / Math.Sqrt((19.523744860232437 / (43)) + (32.45480875927564 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -3.0513572398743407 = (73.36524428499378 - 297.22881358915913) / 73.36524428499378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.61204944920195 > 71.00850860672524. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -177.98651840293232 (T) = (0 -142.4263036319985) / Math.Sqrt((2.877120077816177 / (44)) + (1.5422749865419538 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -1.0990745605914038 = (67.85195071482863 - 142.4263036319985) / 67.85195071482863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.1159585755634 > 111.47950763703729. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.84453890294238 (T) = (0 -140.99687529222126) / Math.Sqrt((3.560680875525543 / (44)) + (5.966375180313497 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3237778810138805 = (106.51097689004432 - 140.99687529222126) / 106.51097689004432 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.98178983177036 > 95.9080285890302. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -70.05958828705552 (T) = (0 -124.03975377971112) / Math.Sqrt((3.0980191035397073 / (44)) + (2.0358104897257205 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3448725761195494 = (92.23160318846809 - 124.03975377971112) / 92.23160318846809 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int64>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 220.7475769463776 > 79.43065832747773. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -119.82692656785483 (T) = (0 -221.16997032067806) / Math.Sqrt((18.233276551426428 / (44)) + (15.510419807790303 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.873910585612455 = (76.95784671517359 - 221.16997032067806) / 76.95784671517359 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).LinkedList(Size%3a%20512).html>) 8.27 ms 10.78 ms 1.30 0.06 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).Stack(Size%3a%20512).html>) 8.12 ms 10.75 ms 1.32 0.07 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).Array(Size%3a%20512).html>) 8.17 ms 10.56 ms 1.29 0.06 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).ImmutableArray(Size%3a%20512).html>) 8.23 ms 10.66 ms 1.30 0.09 False
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).ICollection(Size%3a%20512).html>) 8.11 ms 10.63 ms 1.31 0.05 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).ImmutableList(Size%3a%20512).html>) 17.90 ms 19.38 ms 1.08 0.08 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).Queue(Size%3a%20512).html>) 8.18 ms 10.48 ms 1.28 0.06 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).SortedSet(Size%3a%20512).html>) 2.20 ms 2.51 ms 1.14 0.06 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).List(Size%3a%20512).html>) 8.14 ms 10.50 ms 1.29 0.05 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).ImmutableSortedSet(Size%3a%20512).html>) 2.35 ms 2.68 ms 1.14 0.05 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(String).Span(Size%3a%20512).html>) 3.81 ms 5.69 ms 1.49 0.07 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.782350029166667 > 8.706593060156251. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.700205275700384 (T) = (0 -10797530.224044567) / Math.Sqrt((147065042995.71136 / (43)) + (4082284733.841176 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2876913500955427 = (8385185.0237585455 - 10797530.224044567) / 8385185.0237585455 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.747741697916666 > 8.568740459453124. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.302906499150595 (T) = (0 -10537998.583338674) / Math.Sqrt((163423027369.3921 / (44)) + (10142620044.997364 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2749992524408354 = (8265101.774110786 - 10537998.583338674) / 8265101.774110786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.56096663125 > 8.593167883359374. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.37660640822884 (T) = (0 -10593634.830885742) / Math.Sqrt((23285486736.973618 / (44)) + (6423854706.291151 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2880915275754869 = (8224287.330594926 - 10593634.830885742) / 8224287.330594926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.657508362500002 > 8.691164669801683. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.5446991377658 (T) = (0 -10673708.998970926) / Math.Sqrt((249245459728.3858 / (44)) + (8417137444.9285555 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27943470454754565 = (8342519.521342463 - 10673708.998970926) / 8342519.521342463 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.633040901442309 > 8.57674092203125. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.41955895350131 (T) = (0 -10577676.363043346) / Math.Sqrt((19226146940.175903 / (43)) + (12632700170.306576 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.29110028565569646 = (8192761.228978724 - 10577676.363043346) / 8192761.228978724 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.380178577380956 > 17.742451958942308. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.34783710191634 (T) = (0 -19688885.52083481) / Math.Sqrt((83349115359.6926 / (44)) + (432385753381.32965 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17031488607316916 = (16823579.495684415 - 19688885.52083481) / 16823579.495684415 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.476988016826924 > 8.59866328453125. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.79989477243694 (T) = (0 -10604841.063263126) / Math.Sqrt((18902377603.22366 / (44)) + (16495144056.19459 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2951533064331283 = (8188097.123782989 - 10604841.063263126) / 8188097.123782989 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.505421556122449 > 2.3234903142578123. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.82463750015682 (T) = (0 -2519461.3532235525) / Math.Sqrt((1256579208.2297497 / (44)) + (415528785.4549164 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15004733469683987 = (2190745.786900762 - 2519461.3532235525) / 2190745.786900762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.503750002403846 > 8.596544983569713. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.07701300130955 (T) = (0 -10570918.263651557) / Math.Sqrt((14459335868.973608 / (44)) + (9944895935.188139 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2880760670915574 = (8206750.000037202 - 10570918.263651557) / 8206750.000037202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.68098809375 > 2.4714612719866076. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.38636249866138 (T) = (0 -2740377.806758114) / Math.Sqrt((1535219489.2575545 / (44)) + (2866556847.8921366 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16179672867254674 = (2358741.197256798 - 2740377.806758114) / 2358741.197256798 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsTrue<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.691358977564103 > 3.9783132482812498. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -92.11034062044995 (T) = (0 -5509765.179383903) / Math.Sqrt((6074953331.180068 / (44)) + (3243777218.057316 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4591775074116893 = (3775938.946014324 - 5509765.179383903) / 3775938.946014324 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.CtorFromCollection<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).SortedSet(Size%3a%20512).html>) 3.16 ms 3.73 ms 1.18 0.07 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).ImmutableSortedSet(Size%3a%20512).html>) 3.28 ms 3.95 ms 1.20 0.06 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).ImmutableSortedDictionary(Size%3a%20512).html>) 3.44 ms 3.84 ms 1.12 0.06 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).SortedList(Size%3a%20512).html>) 3.40 ms 3.86 ms 1.13 0.06 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollection(String).SortedDictionary(Size%3a%20512).html>) 2.96 ms 3.36 ms 1.13 0.06 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollection<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.730127676785714 > 3.2761636245625003. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.56403032564845 (T) = (0 -3734682.2611399577) / Math.Sqrt((2987941307.945615 / (44)) + (1855028915.5763052 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20856293179763682 = (3090184.352737783 - 3734682.2611399577) / 3090184.352737783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.9453173064903844 > 3.442144127961539. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.51788204255328 (T) = (0 -3926595.445262515) / Math.Sqrt((2501252302.328778 / (44)) + (1497157549.034142 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20106287810702692 = (3269267.177294789 - 3926595.445262515) / 3269267.177294789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8449899542410715 > 3.6449264071875005. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.203699999012812 (T) = (0 -3878974.2122261147) / Math.Sqrt((7062782485.063823 / (44)) + (1264745920.6152484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1181950576728686 = (3468960.2548403684 - 3878974.2122261147) / 3468960.2548403684 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.857249994711539 > 3.56645078165625. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.136748824640534 (T) = (0 -3936840.490623283) / Math.Sqrt((2973692618.172665 / (44)) + (1204464976.6834896 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17466222661612493 = (3351465.9801092143 - 3936840.490623283) / 3351465.9801092143 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CtorFromCollection<String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.357715416666667 > 3.1818776591145825. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.44830577258752 (T) = (0 -3373684.8978501232) / Math.Sqrt((3101938483.097665 / (44)) + (2037813850.9637456 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12690735687645646 = (2993755.3227101546 - 3373684.8978501232) / 2993755.3227101546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(LoginViewModel).SystemTextJsonSourceGen.html>) 19.59 μs 21.39 μs 1.09 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(LoginViewModel).SystemTextJsonReflection.html>) 19.10 μs 22.10 μs 1.16 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.390529963991767 > 20.229728144536473. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.82236558014516 (T) = (0 -21541.931963836087) / Math.Sqrt((52919.12974666797 / (44)) + (39841.096000700454 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11879635617843604 = (19254.560353967026 - 21541.931963836087) / 19254.560353967026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<LoginViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.097109099953187 > 20.118743742083044. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.47282140518801 (T) = (0 -21847.217843886043) / Math.Sqrt((45473.12508943187 / (44)) + (35108.36163728991 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13409862455640614 = (19263.94880553833 - 21847.217843886043) / 19263.94880553833 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Exceptions.Handling

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ThrowAndCatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Exceptions.Handling.ThrowAndCatch(kind%3a%20ReflectionSoftware).html>) 10.93 μs 11.53 μs 1.05 0.06 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Exceptions.Handling*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Exceptions.Handling.ThrowAndCatch(kind: ReflectionSoftware) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.525822776949825 > 11.412980974476264. IsChangePoint: Marked as a change because one of 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.698667289051498 (T) = (0 -11591.334409298566) / Math.Sqrt((45520.802330682636 / (44)) + (33110.162137126594 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06652682941730015 = (10868.30081492795 - 11591.334409298566) / 10868.30081492795 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 436.93 ns 527.85 ns 1.21 0.13 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 442.71 ns 528.98 ns 1.19 0.11 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(%2c%20None%2c%20False)).html>) 4.76 μs 8.89 μs 1.87 0.06 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 4.76 μs 8.99 μs 1.89 0.09 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 4.79 μs 8.94 μs 1.87 0.08 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20None%2c%20True)).html>) 35.91 μs 40.10 μs 1.12 0.14 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 1.49 μs 1.61 μs 1.08 0.08 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 793.54 ns 905.45 ns 1.14 0.06 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 1.31 μs 1.43 μs 1.09 0.03 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 791.97 ns 906.86 ns 1.15 0.07 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 267.03 ns 376.47 ns 1.41 0.13 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20None%2c%20True)).html>) 36.22 μs 40.50 μs 1.12 0.10 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 432.98 ns 523.36 ns 1.21 0.11 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 444.29 ns 530.47 ns 1.19 0.09 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 10.85 μs 15.42 μs 1.42 0.07 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 4.77 μs 9.42 μs 1.97 0.08 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 16.72 μs 24.50 μs 1.47 0.10 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 11.04 μs 15.45 μs 1.40 0.09 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 4.77 μs 8.93 μs 1.87 0.10 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 16.43 μs 26.74 μs 1.63 0.11 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(%2c%20None%2c%20False)).html>) 4.76 μs 10.42 μs 2.19 0.06 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 12.63 μs 14.45 μs 1.14 0.07 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20Ordinal%2c%20False)).html>) 268.65 ns 375.51 ns 1.40 0.09 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20None%2c%20False)).html>) 10.85 μs 15.42 μs 1.42 0.05 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20OrdinalIgnoreCase%2c%20False)).html>) 1.51 μs 1.62 μs 1.07 0.04 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20True)).html>) 40.98 μs 48.91 μs 1.19 0.08 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(%2c%20IgnoreCase%2c%20True)).html>) 42.27 μs 49.53 μs 1.17 0.08 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 425.14 ns 519.44 ns 1.22 0.15 False
[IndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options%3a%20(%2c%20None%2c%20False)).html>) 10.86 μs 15.45 μs 1.42 0.13 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(%2c%20None%2c%20False)).html>) 424.38 ns 519.25 ns 1.22 0.13 False
[IsSuffix_DifferentLastChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 421.12 ns 522.76 ns 1.24 0.10 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 4.79 μs 8.92 μs 1.86 0.08 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20None%2c%20False)).html>) 428.33 ns 534.61 ns 1.25 0.11 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 10.87 μs 15.39 μs 1.42 0.09 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20IgnoreCase%2c%20False)).html>) 16.46 μs 24.35 μs 1.48 0.10 False
[IsPrefix_FirstHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options%3a%20(en-US%2c%20IgnoreNonSpace%2c%20False)).html>) 4.78 μs 9.10 μs 1.90 0.08 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 428.17 ns 524.60 ns 1.23 0.10 False
[IsPrefix_DifferentFirstChar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options%3a%20(%2c%20IgnoreCase%2c%20False)).html>) 435.73 ns 531.26 ns 1.22 0.08 False
[LastIndexOf_Word_NotFound - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 10.88 μs 16.80 μs 1.54 0.07 False
[IsSuffix_SecondHalf - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options%3a%20(en-US%2c%20None%2c%20False)).html>) 4.77 μs 9.24 μs 1.94 0.07 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 527.8475499737556 > 460.91647729109667. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.999866422418407 (T) = (0 -533.7412972522694) / Math.Sqrt((92.2493042967917 / (44)) + (132.78292172338396 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21004323855697027 = (441.0927479655846 - 533.7412972522694) / 441.0927479655846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 528.9754766192449 > 466.13672754837165. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.48294415842495 (T) = (0 -534.7835773093743) / Math.Sqrt((157.55967339267158 / (43)) + (84.42438047018086 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.20200641898248853 = (444.90908606134934 - 534.7835773093743) / 444.90908606134934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.891775978715316 > 5.019780190745819. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -133.23619192421714 (T) = (0 -8965.008349643) / Math.Sqrt((907.0210495117472 / (44)) + (14476.541784328789 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8747765297318335 = (4781.907714049176 - 8965.008349643) / 4781.907714049176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.987863043373723 > 5.0009262100880365. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -181.93561119846453 (T) = (0 -8937.713997755158) / Math.Sqrt((8597.546062476218 / (44)) + (4803.972721932439 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8595913096410288 = (4806.27864381689 - 8937.713997755158) / 4806.27864381689 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.936333919345579 > 5.029672773308765. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -211.74910802049982 (T) = (0 -8958.054534413312) / Math.Sqrt((4082.242478003767 / (44)) + (4343.517987617865 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8594504086833427 = (4817.581847077286 - 8958.054534413312) / 4817.581847077286 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, True)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.100765825835474 > 37.72101228597125. IsChangePoint: Marked as a change because one of 9/30/2022 6:04:40 AM, 10/26/2022 4:56:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.56588670207561 (T) = (0 -40344.06211359546) / Math.Sqrt((108818.3749589245 / (44)) + (52548.07945403184 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11676761897897926 = (36125.74489801253 - 40344.06211359546) / 36125.74489801253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6103477823034553 > 1.565851478892772. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -3.7061116057021533 (T) = (0 -1619.8979418020187) / Math.Sqrt((25255.381229093477 / (44)) + (2310.866780110239 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06579344313781482 = (1519.8985809415926 - 1619.8979418020187) / 1519.8985809415926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 905.4513212128774 > 833.1687262779119. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.02626993784572 (T) = (0 -908.0406766430509) / Math.Sqrt((485.85282617121095 / (44)) + (14.646134143999305 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13930493620151022 = (797.0128521258725 - 908.0406766430509) / 797.0128521258725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4342788800664037 > 1.3781375218245633. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.684822156301394 (T) = (0 -1435.3629468341092) / Math.Sqrt((261.81401668627075 / (43)) + (45.471684777050356 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.09006528795792254 = (1316.7678694943595 - 1435.3629468341092) / 1316.7678694943595 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 906.8584383753504 > 828.2388128842297. IsChangePoint: Marked as a change because one of 10/13/2022 11:37:03 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.570149479664515 (T) = (0 -906.8418679699756) / Math.Sqrt((151.23969413468112 / (44)) + (46.85302504112427 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14772935633230602 = (790.1182129451558 - 906.8418679699756) / 790.1182129451558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 376.4703876947907 > 282.6324424005217. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.91724700688275 (T) = (0 -379.7691538720855) / Math.Sqrt((43.8548903135161 / (44)) + (76.60100943824663 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.4033342532021054 = (270.61917216481703 - 379.7691538720855) / 270.61917216481703 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, True)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.502544399981495 > 37.9974993662993. IsChangePoint: Marked as a change because one of 9/29/2022 2:35:34 PM, 10/26/2022 2:14:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.837347594037116 (T) = (0 -40513.39935752883) / Math.Sqrt((612347.4789241836 / (44)) + (60595.326529831975 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11459628625525402 = (36348.0480395669 - 40513.39935752883) / 36348.0480395669 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 523.3624037573566 > 445.7824866403164. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.6135762341233 (T) = (0 -521.6358569895327) / Math.Sqrt((85.13892781041697 / (44)) + (44.81222738838533 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21505799737779083 = (429.3094305911914 - 521.6358569895327) / 429.3094305911914 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 530.4651641321237 > 463.6304469703012. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.066968500034804 (T) = (0 -543.6428865525615) / Math.Sqrt((266.94031773151556 / (43)) + (220.65298501811412 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.2219438825405814 = (444.90004354558147 - 543.6428865525615) / 444.90004354558147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.419170792079209 > 11.415516713808662. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.04719149812204 (T) = (0 -15564.880013523583) / Math.Sqrt((30850.173676727904 / (44)) + (156042.4319438624 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42491285803291623 = (10923.390806516272 - 15564.880013523583) / 10923.390806516272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.418783742943052 > 5.00811723924793. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -84.28788230243171 (T) = (0 -9088.229184632868) / Math.Sqrt((5380.466642535488 / (44)) + (36963.09836317792 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8927681340797287 = (4801.554411761904 - 9088.229184632868) / 4801.554411761904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.496847193877546 > 17.287390553104142. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.62614872473061 (T) = (0 -25122.678845619652) / Math.Sqrt((86289.28293345323 / (44)) + (575440.0837637872 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.516793951530563 = (16563.013598695405 - 25122.678845619652) / 16563.013598695405 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.445640265450365 > 11.444584585162419. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -116.93576928080641 (T) = (0 -15420.922673679783) / Math.Sqrt((22793.456619890792 / (44)) + (14241.745473129182 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40941322895521665 = (10941.377842118845 - 15420.922673679783) / 10941.377842118845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.932021731651375 > 5.020955086502719. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.55274167788876 (T) = (0 -9052.06362720866) / Math.Sqrt((3436.4563438490436 / (44)) + (86478.47073833132 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8836811729393267 = (4805.517917389211 - 9052.06362720866) / 4805.517917389211 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.73578159399462 > 17.265423494860244. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.48811148809549 (T) = (0 -24990.957012616964) / Math.Sqrt((145959.72210953434 / (44)) + (624984.1040910525 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5040425431601185 = (16615.85779356273 - 24990.957012616964) / 16615.85779356273 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.418327478995701 > 5.04105717766107. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.11462759408926 (T) = (0 -9092.76879243051) / Math.Sqrt((7713.384678843376 / (44)) + (152135.81702860905 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8884522991712884 = (4814.931675224574 - 9092.76879243051) / 4814.931675224574 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.447411192129628 > 13.013400566350313. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -82.14423298076483 (T) = (0 -14483.370044587075) / Math.Sqrt((21541.35038894785 / (44)) + (1841.1889035360823 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16325755626735236 = (12450.699302621466 - 14483.370044587075) / 12450.699302621466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 375.5093688162158 > 280.7826395326529. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.01056997895808 (T) = (0 -374.728189585504) / Math.Sqrt((59.30784147547476 / (44)) + (102.88386937597663 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39458696172766855 = (268.70191667450854 - 374.728189585504) / 268.70191667450854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.419170787953798 > 11.397556962150567. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.46138846278189 (T) = (0 -15458.936904984805) / Math.Sqrt((709411.196556569 / (44)) + (32242.87415094633 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3963524961540739 = (11070.941576401967 - 15458.936904984805) / 11070.941576401967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, OrdinalIgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6162842393605756 > 1.5774056400541427. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.967601387353245 (T) = (0 -1619.6792049217486) / Math.Sqrt((645.4831742967705 / (44)) + (155.67105704543633 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07282316305982191 = (1509.7354910777904 - 1619.6792049217486) / 1509.7354910777904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, True)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.908161913218855 > 43.851624756505515. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 10:22:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.05538595124943 (T) = (0 -49415.61961495066) / Math.Sqrt((676153.7395257549 / (44)) + (655312.920489697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18024926645088957 = (41868.79926097955 - 49415.61961495066) / 41868.79926097955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, True)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.526360345804996 > 44.00717298559004. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.38293271059544 (T) = (0 -49887.59569354359) / Math.Sqrt((1012894.9070926773 / (44)) + (961077.4804117514 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19329720089625063 = (41806.513629692985 - 49887.59569354359) / 41806.513629692985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 519.4358811976015 > 451.70694545230486. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.51859408428911 (T) = (0 -523.8099303954579) / Math.Sqrt((133.36728830721756 / (44)) + (77.24738083178093 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2078827388304175 = (433.6595876042227 - 523.8099303954579) / 433.6595876042227 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.45262674571805 > 11.481722502622965. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.079124605990696 (T) = (0 -15647.86144786941) / Math.Sqrt((754388.0267614659 / (44)) + (102508.87068385405 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40403733359823635 = (11144.904108616123 - 15647.86144786941) / 11144.904108616123 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 519.2491501164404 > 448.57386417071706. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.18045433976721 (T) = (0 -524.9376603496146) / Math.Sqrt((132.7610903170276 / (43)) + (130.85967102492563 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.21647253927088195 = (431.5244638931569 - 524.9376603496146) / 431.5244638931569 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 522.7591954355884 > 450.7027311893214. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.124630868061764 (T) = (0 -523.9441588704257) / Math.Sqrt((86.00953353759044 / (44)) + (29.48324353951502 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2166683451232918 = (430.6384405993003 - 523.9441588704257) / 430.6384405993003 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.922697148243603 > 5.027631584311033. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -145.57877654666586 (T) = (0 -8963.097610546807) / Math.Sqrt((4328.905993263638 / (44)) + (10731.338169613247 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8633801280510413 = (4810.1283659827095 - 8963.097610546807) / 4810.1283659827095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 534.6143745901547 > 453.0596767165984. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.08163859652046 (T) = (0 -525.7125027566378) / Math.Sqrt((121.00554587828155 / (44)) + (64.57374920091263 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20801480148136203 = (435.1871368728 - 525.7125027566378) / 435.1871368728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.390558288177338 > 11.408879791731469. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.055856376909226 (T) = (0 -15559.11035151324) / Math.Sqrt((333411.9071939683 / (44)) + (63211.9599683674 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4154570700952049 = (10992.287000598839 - 15559.11035151324) / 10992.287000598839 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.353317228115976 > 17.290033556725508. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.78025456644539 (T) = (0 -24885.67850153552) / Math.Sqrt((73499.05260838443 / (44)) + (540479.8505559043 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5010371883016262 = (16578.988645639645 - 24885.67850153552) / 16578.988645639645 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.09724969653374 > 5.02335957421502. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -196.36517865494616 (T) = (0 -8964.856314797838) / Math.Sqrt((5549.108401258905 / (44)) + (4804.030669242578 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8614394700706764 = (4816.088010886249 - 8964.856314797838) / 4816.088010886249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 524.6027796123091 > 452.30446596920797. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.0389693394159 (T) = (0 -522.1694708636273) / Math.Sqrt((91.33981088916599 / (44)) + (10.39696417016805 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20340391058666052 = (433.91039888599767 - 522.1694708636273) / 433.91039888599767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 531.2568925079007 > 461.214163576132. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.388257274276526 (T) = (0 -531.9910632126231) / Math.Sqrt((76.44680656533356 / (44)) + (56.30129303092507 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19715127370370986 = (444.3808187805418 - 531.9910632126231) / 444.3808187805418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.800539085343566 > 11.390643346482825. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.53741714456128 (T) = (0 -15544.181907859635) / Math.Sqrt((40151.3208531223 / (44)) + (153569.42831171522 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.41997371457315874 = (10946.809612269608 - 15544.181907859635) / 10946.809612269608 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, None, False)) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.241444552354139 > 5.017508589866255. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -147.35660671756733 (T) = (0 -8994.6248017504) / Math.Sqrt((9474.742950184616 / (44)) + (8835.572873562429 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8678974088892403 = (4815.37409867661 - 8994.6248017504) / 4815.37409867661 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).ToArray(Size%3a%20512).html>) 707.58 ns 998.85 ns 1.41 0.05 False
[LastIndexOfAnyValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).LastIndexOfAnyValues(Size%3a%20512).html>) 7.70 μs 8.80 μs 1.14 0.09 False
[Fill - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).Fill(Size%3a%20512).html>) 3.54 μs 4.31 μs 1.21 0.06 False
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).Reverse(Size%3a%20512).html>) 3.72 μs 4.04 μs 1.09 0.11 False
[IndexOfAnyFiveValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).IndexOfAnyFiveValues(Size%3a%20512).html>) 6.48 μs 16.30 μs 2.51 0.06 False
[StartsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).StartsWith(Size%3a%20512).html>) 1.35 μs 1.44 μs 1.07 0.05 False
[BinarySearch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).BinarySearch(Size%3a%20512).html>) 339.46 ns 401.31 ns 1.18 0.08 False
[SequenceCompareToDifferent - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Char).SequenceCompareToDifferent(Size%3a%20512).html>) 145.07 ns 200.17 ns 1.38 0.12 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 998.8528422358787 > 762.8917687266962. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.527876278915455 (T) = (0 -1008.3502664763902) / Math.Sqrt((26372.063177123964 / (44)) + (543.7165364054566 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35726484877413855 = (742.9281524435833 - 1008.3502664763902) / 742.9281524435833 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.798853517194278 > 8.086584048832831. IsChangePoint: Marked as a change because one of 9/23/2022 2:49:56 PM, 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.306671695963447 (T) = (0 -8642.372317866357) / Math.Sqrt((143873.3292203398 / (44)) + (11783.251526173673 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10871047364738044 = (7794.976707882186 - 8642.372317866357) / 7794.976707882186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Fill(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.3056040304331775 > 3.723598843546615. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.68795639106694 (T) = (0 -4389.105974287566) / Math.Sqrt((1292.509764022836 / (44)) + (5236.47170501393 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23336688213214463 = (3558.6377726472074 - 4389.105974287566) / 3558.6377726472074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.Reverse(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.042685566250434 > 3.9024582931024563. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.233290405303471 (T) = (0 -4088.834446961537) / Math.Sqrt((14727.32903946772 / (44)) + (20244.0581653096 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09008354440809727 = (3750.9367680453583 - 4088.834446961537) / 3750.9367680453583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.29829164647331 > 6.799278912407642. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -128.4142568795551 (T) = (0 -16374.100140827137) / Math.Sqrt((204993.87359458374 / (44)) + (17113.054546596355 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.4829889852282263 = (6594.511791328827 - 16374.100140827137) / 6594.511791328827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.StartsWith(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4382461168116656 > 1.407819771222538. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.893567317897272 (T) = (0 -1443.142798023355) / Math.Sqrt((26.947809866721943 / (44)) + (170.03567275695622 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.07415685369908179 = (1343.512163101314 - 1443.142798023355) / 1343.512163101314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.BinarySearch(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 401.31176820129457 > 356.34140786646975. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.0913468865954 (T) = (0 -399.2725959623244) / Math.Sqrt((34.557049075576856 / (44)) + (27.290135446677237 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16528969710772104 = (342.6380555438954 - 399.2725959623244) / 342.6380555438954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.16556833876007 > 150.54954708950643. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.84811056696568 (T) = (0 -201.77868642000232) / Math.Sqrt((49.33700992329782 / (44)) + (5.364021075479096 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38182963367284456 = (146.02283921476132 - 201.77868642000232) / 146.02283921476132 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromReader(Mode%3a%20SourceGen).html>) 3.17 μs 3.54 μs 1.12 0.05 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromString(Mode%3a%20Reflection).html>) 3.28 μs 3.95 μs 1.20 0.12 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 1.67 μs 1.91 μs 1.14 0.06 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromReader(Mode%3a%20Reflection).html>) 3.13 μs 3.62 μs 1.16 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.65 μs 1.85 μs 1.13 0.08 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Int32).DeserializeFromString(Mode%3a%20SourceGen).html>) 3.23 μs 3.97 μs 1.23 0.07 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5392724764442387 > 3.3233834849314827. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.648033558696312 (T) = (0 -3616.440559995452) / Math.Sqrt((4474.012266824433 / (44)) + (3938.0079020529806 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14258795024892515 = (3165.1310161354068 - 3616.440559995452) / 3165.1310161354068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.9454248481230096 > 3.402922244267054. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.090151072742177 (T) = (0 -3930.0613993853563) / Math.Sqrt((4661.924012566145 / (44)) + (6442.195051989365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20667578656870583 = (3256.9323451503483 - 3930.0613993853563) / 3256.9323451503483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.906082495519153 > 1.730860967413865. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.320427913859312 (T) = (0 -1888.6247148153316) / Math.Sqrt((1672.2262330451317 / (44)) + (405.84116957346106 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12125224804009456 = (1684.3887877295892 - 1888.6247148153316) / 1684.3887877295892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.621161377280074 > 3.3186132606260257. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.242441013039272 (T) = (0 -3604.1873119631773) / Math.Sqrt((2720.8969697733137 / (44)) + (2555.0253471490614 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1410664044359454 = (3158.613116600175 - 3604.1873119631773) / 3158.613116600175 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.853172341008772 > 1.7427547047021479. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.31780484676668 (T) = (0 -1881.156084628268) / Math.Sqrt((2417.3176230137 / (44)) + (1542.2861721508527 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11685433923891697 = (1684.3343115899843 - 1881.156084628268) / 1684.3343115899843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Int32>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.968512210126256 > 3.3862331376931754. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.50722711999204 (T) = (0 -3941.17300419153) / Math.Sqrt((3533.660338709556 / (44)) + (4566.333791305661 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21464464448149106 = (3244.712782538915 - 3941.17300419153) / 3244.712782538915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array_ComparerClass(Size%3a%20512).html>) 450.44 μs 569.95 μs 1.27 0.04 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).List(Size%3a%20512).html>) 456.35 μs 674.92 μs 1.48 0.02 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array_Comparison(Size%3a%20512).html>) 446.56 μs 566.40 μs 1.27 0.02 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array_ComparerStruct(Size%3a%20512).html>) 471.33 μs 603.11 μs 1.28 0.05 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(BigStruct).Array(Size%3a%20512).html>) 440.04 μs 651.66 μs 1.48 0.04 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 569.9528666133334 > 478.25166524800005. IsChangePoint: Marked as a change because one of 10/23/2022 1:14:54 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.143527171181264 (T) = (0 -570137.3620804909) / Math.Sqrt((2184316329.365362 / (44)) + (20467907.345881034 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21532240393921243 = (469124.37410230434 - 570137.3620804909) / 469124.37410230434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<BigStruct>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 674.9209332933332 > 487.2263830359999. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.144931339964664 (T) = (0 -658694.0344774066) / Math.Sqrt((1990139259.2618024 / (44)) + (58926302.07396168 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42772118176041085 = (461360.4132882744 - 658694.0344774066) / 461360.4132882744 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 566.399342857143 > 471.03783002100005. IsChangePoint: Marked as a change because one of 10/23/2022 1:14:54 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.137623496601986 (T) = (0 -570356.0673574138) / Math.Sqrt((2096680182.7499151 / (44)) + (106541886.63636039 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2246159683700553 = (465742.797814852 - 570356.0673574138) / 465742.797814852 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<BigStruct>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 603.1113600266667 > 496.57670002099997. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.350824654471415 (T) = (0 -599385.9011454018) / Math.Sqrt((2113857602.2082825 / (44)) + (18316433.77058203 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2191516407228215 = (491641.7951010856 - 599385.9011454018) / 491641.7951010856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<BigStruct>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 651.6622800399999 > 477.04940247999997. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.278785707324033 (T) = (0 -653615.8629800976) / Math.Sqrt((1956954758.004176 / (44)) + (26526107.551111676 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.41686767703750444 = (461310.4480911921 - 653615.8629800976) / 461310.4480911921 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.AddGivenSize(Int32).ConcurrentDictionary(Size%3a%20512).html>) 330.41 μs 364.57 μs 1.10 0.04 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.AddGivenSize(Int32).HashSet(Size%3a%20512).html>) 76.82 μs 92.13 μs 1.20 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.AddGivenSize<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.56763556201554 > 348.3340358409626. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.348899854127765 (T) = (0 -369479.6980047556) / Math.Sqrt((33930966.92521411 / (44)) + (23329957.081667747 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09665038773546092 = (336916.57992089563 - 369479.6980047556) / 336916.57992089563 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.AddGivenSize<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.13058894710578 > 80.10412843781386. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.66117860848488 (T) = (0 -92737.88950182607) / Math.Sqrt((2429963.968180702 / (43)) + (936742.7213462971 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.21471800775024938 = (76345.20021118624 - 92737.88950182607) / 76345.20021118624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(IndexViewModel).SystemTextJsonSourceGen.html>) 1.11 ms 1.25 ms 1.12 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromString(IndexViewModel).JsonNet.html>) 1.73 ms 1.87 ms 1.08 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2480990687645688 > 1.1662159396314498. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.039574563124575 (T) = (0 -1227657.8533391124) / Math.Sqrt((142339184.40452078 / (44)) + (92903523.90403786 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1051480476963164 = (1110853.7502266488 - 1227657.8533391124) / 1110853.7502266488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8730682078410308 > 1.765289384075342. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.00865949484527 (T) = (0 -1874442.2060908754) / Math.Sqrt((462800700.49892676 / (44)) + (216653082.76995042 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11936932721786504 = (1674552.054012151 - 1874442.2060908754) / 1674552.054012151 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Object>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[RentNoReturn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.NonStandardArrayPoolTests(Object).RentNoReturn(RentalSize%3a%2064%2c%20UseSharedPool%3a%20True).html>) 228.38 ns 244.30 ns 1.07 0.10 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Object&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.NonStandardArrayPoolTests<Object>.RentNoReturn(RentalSize: 64, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 244.29835742857145 > 241.02484080681822. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/12/2022 2:17:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.130154696714479 (T) = (0 -245.27088430014646) / Math.Sqrt((30.59601716312829 / (44)) + (5.580780414151221 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06807137322661415 = (229.6390395327139 - 245.27088430014646) / 229.6390395327139 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).AndNotBenchmark.html>) 132.95 ns 266.66 ns 2.01 0.05 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GreaterThanAnyBenchmark.html>) 72.12 ns 251.91 ns 3.49 0.09 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).OnesComplementOperatorBenchmark.html>) 104.22 ns 151.79 ns 1.46 0.12 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).EqualsAnyBenchmark.html>) 212.09 ns 279.11 ns 1.32 0.17 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).AllBitsSetBenchmark.html>) 43.57 ns 88.38 ns 2.03 0.11 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).LessThanOrEqualAnyBenchmark.html>) 217.64 ns 290.53 ns 1.33 0.17 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).ConditionalSelectBenchmark.html>) 179.99 ns 417.24 ns 2.32 0.04 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GreaterThanOrEqualAnyBenchmark.html>) 72.44 ns 259.32 ns 3.58 0.13 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).LessThanAllBenchmark.html>) 75.51 ns 322.62 ns 4.27 0.07 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).OnesComplementBenchmark.html>) 101.64 ns 153.11 ns 1.51 0.12 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).LessThanOrEqualAllBenchmark.html>) 72.17 ns 345.98 ns 4.79 0.07 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GetHashCodeBenchmark.html>) 717.20 ns 866.09 ns 1.21 0.01 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GreaterThanOrEqualAllBenchmark.html>) 202.18 ns 346.42 ns 1.71 0.10 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).GreaterThanAllBenchmark.html>) 198.34 ns 343.83 ns 1.73 0.04 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt32).SumBenchmark.html>) 66.24 ns 85.56 ns 1.29 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.66349736875213 > 139.84665507333062. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.62628954753295 (T) = (0 -266.83896076272737) / Math.Sqrt((954.495712647135 / (44)) + (25.923877568188484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7497853594585262 = (152.49811030839868 - 266.83896076272737) / 152.49811030839868 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 251.90984723198545 > 75.77096355127223. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -189.9249743913848 (T) = (0 -254.17167443845224) / Math.Sqrt((7.542988190405581 / (44)) + (10.410431844486313 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -2.5060879186048908 = (72.49438129879807 - 254.17167443845224) / 72.49438129879807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.79275332186486 > 109.01943386845167. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.800738527824976 (T) = (0 -154.47260092325678) / Math.Sqrt((1008.0600092032955 / (44)) + (5.309503864055386 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26959824637655544 = (121.67045863848894 - 154.47260092325678) / 121.67045863848894 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.1148531861885 > 214.09018462428693. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.40865231697607 (T) = (0 -278.4809650020028) / Math.Sqrt((34.80603218052882 / (44)) + (11.664329486346 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39020605526685104 = (200.3163228551383 - 278.4809650020028) / 200.3163228551383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.37887593815472 > 46.52195786422347. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.467996102692281 (T) = (0 -87.83255272544689) / Math.Sqrt((934.4666910247405 / (44)) + (6.852647990176437 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40838678829124847 = (62.36394253031255 - 87.83255272544689) / 62.36394253031255 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 290.5267433716155 > 215.21957294052604. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.442828394841797 (T) = (0 -286.48038974733595) / Math.Sqrt((203.5369581456314 / (44)) + (25.55381445058838 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3814578577577767 = (207.3754100702847 - 286.48038974733595) / 207.3754100702847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.2443374978752 > 190.1663112141101. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.64619760152963 (T) = (0 -421.2404549339305) / Math.Sqrt((879.6269596947587 / (43)) + (23.351269643877714 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -1.1307810023129246 = (197.69298415777197 - 421.2404549339305) / 197.69298415777197 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 259.3208330798776 > 76.48666400484187. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -146.80504792082112 (T) = (0 -260.79280456208824) / Math.Sqrt((1.0540166161453355 / (44)) + (24.03215174697845 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5439044706561966 = (73.58911808189889 - 260.79280456208824) / 73.58911808189889 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.61773590693235 > 75.41166721810335. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -658.246472588398 (T) = (0 -320.8047993923106) / Math.Sqrt((2.0050778039569734 / (44)) + (1.3659840597090362 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -3.4702358252405743 = (71.76462538753107 - 320.8047993923106) / 71.76462538753107 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.1061540381962 > 108.16076353042712. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.890010549680192 (T) = (0 -153.78100848054612) / Math.Sqrt((898.8068866971097 / (44)) + (8.35824799475974 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25824596902304037 = (122.21855842697332 - 153.78100848054612) / 122.21855842697332 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 345.98394771955367 > 77.09262090113064. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -162.52612822687604 (T) = (0 -330.35368454253876) / Math.Sqrt((16.357892391104134 / (44)) + (31.767541132433344 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.4695509330143603 = (73.91205279760426 - 330.35368454253876) / 73.91205279760426 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 866.0909327764782 > 750.1289297148985. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.44870831076038 (T) = (0 -841.6843956163876) / Math.Sqrt((108.40039064103387 / (44)) + (77.31556183228449 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.17079637852552781 = (718.8990426126738 - 841.6843956163876) / 718.8990426126738 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 346.42424551196655 > 212.52301785611346. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -171.36671833467594 (T) = (0 -345.8162977129916) / Math.Sqrt((6.230227886762555 / (44)) + (8.192057794310731 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6976543661929941 = (203.70241705235202 - 345.8162977129916) / 203.70241705235202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 343.83253579303727 > 206.72927460084702. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.21469717271752 (T) = (0 -343.0252670884263) / Math.Sqrt((10.325754971899089 / (44)) + (44.44246144186062 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7341972811601676 = (197.80060251215735 - 343.0252670884263) / 197.80060251215735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 85.56353454924134 > 69.55871762589912. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.50856849838207 (T) = (0 -81.18527369007974) / Math.Sqrt((1.2168407963680068 / (44)) + (2.3560428656238694 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21565032711900264 = (66.78340957015375 - 81.18527369007974) / 66.78340957015375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Net.Primitives.Tests.CredentialCacheTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%200%2c%20hostPortCount%3a%2010).html>) 3.67 μs 4.02 μs 1.09 0.04 False
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%2010%2c%20hostPortCount%3a%200).html>) 3.69 μs 4.06 μs 1.10 0.02 False
[ForEach - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount%3a%2010%2c%20hostPortCount%3a%2010).html>) 6.66 μs 7.74 μs 1.16 0.02 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.CredentialCacheTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 0, hostPortCount: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.020934513045725 > 3.8646707466330277. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.975279911122398 (T) = (0 -4073.78842343449) / Math.Sqrt((8978.473237029753 / (44)) + (4488.594136706301 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0898808480394524 = (3737.829167989035 - 4073.78842343449) / 3737.829167989035 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.063732527368872 > 3.959335520831087. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/17/2022 4:58:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.870369271079074 (T) = (0 -4078.7461119618065) / Math.Sqrt((5194.173447674475 / (44)) + (3120.35844627545 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0911589604882871 = (3737.9944257952957 - 4078.7461119618065) / 3737.9944257952957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.CredentialCacheTests.ForEach(uriCount: 10, hostPortCount: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.735764229636899 > 7.091929554596894. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.817075655482544 (T) = (0 -7466.358489166855) / Math.Sqrt((9480.171887363862 / (44)) + (10413.103862774076 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10152843429875924 = (6778.180441542566 - 7466.358489166855) / 6778.180441542566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Type

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[op_Equality - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Type.op_Equality.html>) 19.09 ns 32.19 ns 1.69 0.38 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Type*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Type.op_Equality ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.19496891503763 > 20.83076264987962. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.716460201290275 (T) = (0 -32.585444353753324) / Math.Sqrt((1.4949958986311236 / (44)) + (1.1490378630257398 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5992128954009759 = (20.375926461988083 - 32.585444353753324) / 20.375926461988083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Sort<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).Array_ComparerClass(Size%3a%20512).html>) 2.88 ms 3.37 ms 1.17 0.07 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).LinqOrderByExtension(Size%3a%20512).html>) 2.73 ms 3.17 ms 1.16 0.09 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).Array_Comparison(Size%3a%20512).html>) 2.86 ms 3.34 ms 1.17 0.06 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).Array_ComparerStruct(Size%3a%20512).html>) 2.90 ms 3.36 ms 1.16 0.07 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).List(Size%3a%20512).html>) 2.73 ms 3.29 ms 1.21 0.07 False
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(String).LinqQuery(Size%3a%20512).html>) 2.66 ms 3.16 ms 1.18 0.06 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<String>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3724140000133334 > 3.0010747480210007. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.67287545284168 (T) = (0 -3320750.670123498) / Math.Sqrt((3125336909.20552 / (44)) + (753989295.1324484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1648590800584935 = (2850774.5932295485 - 3320750.670123498) / 2850774.5932295485 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<String>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1674992166000004 > 2.7862026195214997. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.39245537198481 (T) = (0 -3162148.685536211) / Math.Sqrt((1355339467.1941366 / (44)) + (3488343591.2071943 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1920487237432355 = (2652700.869144448 - 3162148.685536211) / 2652700.869144448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<String>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.33984630696 > 3.0093381764425. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.13901352230742 (T) = (0 -3341023.34310034) / Math.Sqrt((2448838198.4240727 / (44)) + (386262656.3485275 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.17349770078420082 = (2847064.2429615925 - 3341023.34310034) / 2847064.2429615925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<String>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3643729598133336 > 3.0487801119789997. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.825424359572516 (T) = (0 -3375971.7831618413) / Math.Sqrt((2395557674.9096828 / (44)) + (2095901565.604145 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16787519129513184 = (2890695.691050693 - 3375971.7831618413) / 2890695.691050693 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2932017428428573 > 2.8855012901190005. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.41425076014076 (T) = (0 -3300600.1900158925) / Math.Sqrt((1242142997.6340215 / (44)) + (1292083340.1445315 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20929066923230127 = (2729368.772944577 - 3300600.1900158925) / 2729368.772944577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<String>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.157594960026667 > 2.775623725007. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.23174817249699 (T) = (0 -3120786.1780325198) / Math.Sqrt((1683669711.8254156 / (44)) + (762019442.4747514 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17911291875030277 = (2646723.76021469 - 3120786.1780325198) / 2646723.76021469 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchF.BenchMk2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchF.BenchMk2.Test.html>) 616.44 ms 683.57 ms 1.11 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.BenchMk2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.BenchMk2.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 683.5714614615384 > 648.516675. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.9086794947618 (T) = (0 -688203165.2308059) / Math.Sqrt((162050989271145.28 / (44)) + (175071370234230.72 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10653020130351999 = (621947023.60594 - 688203165.2308059) / 621947023.60594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyTrue(String%2c%20String).ImmutableSortedDictionary(Size%3a%20512).html>) 2.33 ms 2.65 ms 1.14 0.08 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyTrue(String%2c%20String).SortedDictionary(Size%3a%20512).html>) 2.56 ms 2.87 ms 1.12 0.07 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyTrue(String%2c%20String).ConcurrentDictionary(Size%3a%20512).html>) 247.61 μs 275.15 μs 1.11 0.07 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyTrue(String%2c%20String).SortedList(Size%3a%20512).html>) 2.12 ms 2.46 ms 1.16 0.09 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyTrue<String, String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.6534479184027777 > 2.4193041963701925. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.10439471597517 (T) = (0 -2640024.06574303) / Math.Sqrt((1474404407.0539784 / (44)) + (694940992.0437425 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1525385336639321 = (2290616.746105977 - 2640024.06574303) / 2290616.746105977 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<String, String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.873347473214286 > 2.6755928125. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.948892251748887 (T) = (0 -2874933.851172288) / Math.Sqrt((2308449974.603928 / (44)) + (1074276692.3124328 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1362539791191802 = (2530185.9478643374 - 2874933.851172288) / 2530185.9478643374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<String, String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 275.1492687134503 > 259.58837519856775. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.96149076013868 (T) = (0 -268158.42178962566) / Math.Sqrt((15844170.188455107 / (44)) + (39331392.488577195 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08345913609950305 = (247502.10954425737 - 268158.42178962566) / 247502.10954425737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyTrue<String, String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4570488077380954 > 2.257777501171875. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.28343628022946 (T) = (0 -2499699.2125071953) / Math.Sqrt((1219450100.8552318 / (44)) + (1504624393.618971 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16496140667348003 = (2145735.642560922 - 2499699.2125071953) / 2145735.642560922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadToEndTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 287.25 ms 344.96 ms 1.20 0.04 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%208%5d).html>) 246.18 ms 345.73 ms 1.40 0.04 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 246.62 ms 335.90 ms 1.36 0.03 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%201%2c%20%20%20%201%5d).html>) 253.79 ms 336.72 ms 1.33 0.02 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%20%20%20%200%5d).html>) 246.40 ms 334.57 ms 1.36 0.03 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) 246.82 ms 338.82 ms 1.37 0.02 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 265.26 ms 347.98 ms 1.31 0.04 False
[ReadToEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 250.45 ms 340.48 ms 1.36 0.02 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadToEndTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 344.95878564285715 > 284.3796069975. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.300114002066438 (T) = (0 -339440402.9126252) / Math.Sqrt((2040209367196244 / (44)) + (15957788674224.875 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2641987085370926 = (268502412.3347028 - 339440402.9126252) / 268502412.3347028 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 8]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 345.7326501 > 262.8104120976923. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.981760387642751 (T) = (0 -340610181.0577778) / Math.Sqrt((1930780211529564.2 / (44)) + (27702162848324.684 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27881325983281646 = (266348646.63688812 - 340610181.0577778) / 266348646.63688812 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 335.89615369230773 > 264.4926174288462. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.990567333602629 (T) = (0 -338828650.1195604) / Math.Sqrt((1899757480239393.5 / (44)) + (10390354956096.697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2736287212873308 = (266034083.91818187 - 338828650.1195604) / 266034083.91818187 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 1, 1]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 336.7209170833333 > 275.70871151250003. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.295641198552723 (T) = (0 -337643145.8868393) / Math.Sqrt((1946335834702613.8 / (43)) + (18489857335366.027 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.2628187923549446 = (267372601.62021473 - 337643145.8868393) / 267372601.62021473 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 0]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 334.5713333333333 > 266.1609785443269. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.49645054157043 (T) = (0 -337488248.9022589) / Math.Sqrt((1861835500752948.5 / (43)) + (28346998055208.54 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.26433830817657794 = (266928753.73600176 - 337488248.9022589) / 266928753.73600176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 338.81830761538464 > 265.3298306076923. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.254560352395462 (T) = (0 -339917241.60641026) / Math.Sqrt((1831639933045212.2 / (44)) + (10083759749488.822 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27443977432190814 = (266718952.48033217 - 339917241.60641026) / 266718952.48033217 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.9842857857143 > 272.275447395. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.977114346762049 (T) = (0 -342066373.5515873) / Math.Sqrt((1901708019979098.8 / (44)) + (33776573448443.195 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27615686522423044 = (268044143.21861884 - 342066373.5515873) / 268044143.21861884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadToEndTests.ReadToEnd(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.4760002142857 > 270.01192187509616. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.077222782467143 (T) = (0 -340181213.3230892) / Math.Sqrt((1891666766190286.5 / (44)) + (21415710298173.652 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27718060901302105 = (266353255.69652537 - 340181213.3230892) / 266353255.69652537 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 16.81 μs 18.90 μs 1.12 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20SourceGen).html>) 8.64 μs 10.41 μs 1.20 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20Reflection).html>) 8.66 μs 10.56 μs 1.22 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 6.95 μs 8.61 μs 1.24 0.04 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20Reflection).html>) 3.38 μs 4.66 μs 1.38 0.06 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20SourceGen).html>) 3.59 μs 4.88 μs 1.36 0.08 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 16.07 μs 18.48 μs 1.15 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 6.90 μs 8.41 μs 1.22 0.01 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) graph_6.png>) graph_7.png>) graph_8.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.898808870640714 > 17.421787571143618. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.83637214501085 (T) = (0 -18871.438283686046) / Math.Sqrt((31617.200623451223 / (44)) + (38205.81240070932 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13717873443397566 = (16594.962350469206 - 18871.438283686046) / 16594.962350469206 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.410442764037434 > 9.043091035294132. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.27869855707847 (T) = (0 -10366.192060013791) / Math.Sqrt((30952.342748682215 / (43)) + (13668.203657403592 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.20846977426500457 = (8577.948973790879 - 10366.192060013791) / 8577.948973790879 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.564739696174676 > 9.09204926912242. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.95102578027574 (T) = (0 -10366.780101543804) / Math.Sqrt((24424.849263117598 / (43)) + (6727.559127826627 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.207841845991971 = (8582.895298705122 - 10366.780101543804) / 8582.895298705122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.614723031868738 > 7.289249380608584. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.601006948501265 (T) = (0 -8589.13127287444) / Math.Sqrt((17786.624370579317 / (44)) + (8391.509909632072 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24585871355143107 = (6894.145523444113 - 8589.13127287444) / 6894.145523444113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.660330272556391 > 3.7865820786267292. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.687248350513414 (T) = (0 -4742.879326956235) / Math.Sqrt((10264.178754785366 / (44)) + (7258.19248689081 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.33746721678411074 = (3546.164920857132 - 4742.879326956235) / 3546.164920857132 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.882609222449193 > 3.835422166435402. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.98785882624093 (T) = (0 -4745.016896863556) / Math.Sqrt((15397.961495554982 / (44)) + (5903.401054322884 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32807702076392437 = (3572.8476757576686 - 4745.016896863556) / 3572.8476757576686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.478736153998415 > 17.067055414882372. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.88238266338859 (T) = (0 -18468.27415832427) / Math.Sqrt((30869.24325444627 / (44)) + (15543.803045571663 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13554813654462397 = (16263.752776277413 - 18468.27415832427) / 16263.752776277413 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.410672181703584 > 7.339163504700463. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.82644724095858 (T) = (0 -8539.705026230025) / Math.Sqrt((24436.514934658186 / (44)) + (10275.922404571249 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23934740792537165 = (6890.48524378263 - 8539.705026230025) / 6890.48524378263 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(IndexViewModel).DataContractSerializer.html>) 2.03 ms 2.30 ms 1.13 0.01 False
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(IndexViewModel).XmlSerializer.html>) 1.40 ms 1.60 ms 1.15 0.02 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2980166909212283 > 2.1501331982581964. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.87516222002066 (T) = (0 -2303133.4100878406) / Math.Sqrt((607437177.6496911 / (44)) + (420212642.5322463 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12965052037615854 = (2038801.7077360598 - 2303133.4100878406) / 2038801.7077360598 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_ToStream<IndexViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5985946433035716 > 1.462460792471591. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.387663922701186 (T) = (0 -1606995.7748441587) / Math.Sqrt((175617451.22202885 / (44)) + (172779351.84527522 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14841265821663602 = (1399319.1065480364 - 1606995.7748441587) / 1399319.1065480364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NewCachedCustomConverter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).NewCachedCustomConverter.html>) 55.45 μs 59.16 μs 1.07 0.01 False
[CachedJsonSerializerContext - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).CachedJsonSerializerContext.html>) 29.96 μs 32.28 μs 1.08 0.01 False
[CachedDefaultOptions - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ColdStartSerialization(SimpleStructWithProperties).CachedDefaultOptions.html>) 32.31 μs 35.05 μs 1.08 0.00 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCachedCustomConverter ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.15537097591888 > 58.270656664898716. IsChangePoint: Marked as a change because one of 10/6/2022 12:41:43 AM, 10/17/2022 12:47:34 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.11646355787732 (T) = (0 -58875.85632981578) / Math.Sqrt((167828.31530575643 / (44)) + (70533.61964742951 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.061772011304553856 = (55450.563494772796 - 58875.85632981578) / 55450.563494772796 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedJsonSerializerContext ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.281296090982885 > 31.264628523820885. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.879709032908295 (T) = (0 -32130.768556571285) / Math.Sqrt((70915.21463873214 / (44)) + (38425.99895745038 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08243971171180058 = (29683.656474279556 - 32130.768556571285) / 29683.656474279556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.CachedDefaultOptions ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.05323017790262 > 34.347361991697184. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.26246499366815 (T) = (0 -35072.91832162389) / Math.Sqrt((67586.9188691062 / (44)) + (35725.319936876804 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07949157898243048 = (32490.219474138878 - 35072.91832162389) / 32490.219474138878 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(CollectionsOfPrimitives).XmlSerializer.html>) 19.77 ms 21.31 ms 1.08 0.01 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(CollectionsOfPrimitives).DataContractSerializer.html>) 54.35 ms 58.53 ms 1.08 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.310350649350646 > 20.386097313076924. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.386937579981623 (T) = (0 -21376094.868075628) / Math.Sqrt((50199668785.20321 / (44)) + (59784138076.10857 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10507325328452245 = (19343599.896695662 - 21376094.868075628) / 19343599.896695662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_FromStream<CollectionsOfPrimitives>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.525160678571424 > 57.81775122375001. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/21/2022 5:53:28 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.387823419901068 (T) = (0 -59142585.384975575) / Math.Sqrt((315928274974.2993 / (44)) + (214308619210.85464 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07852098461015972 = (54836749.7980145 - 59142585.384975575) / 54836749.7980145 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(CollectionsOfPrimitives).XmlSerializer.html>) 11.66 ms 12.83 ms 1.10 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<CollectionsOfPrimitives>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.83063157518797 > 12.249313345000003. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.469670419031637 (T) = (0 -12852172.419405067) / Math.Sqrt((36180300013.91673 / (44)) + (14953040080.200375 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11249217118131866 = (11552595.831535399 - 12852172.419405067) / 11552595.831535399 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Memory<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Memory(Char).ToArray(Size%3a%20512).html>) 735.56 ns 1.06 μs 1.44 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Memory<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0576997207002063 > 779.0500433637928. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.805818857566656 (T) = (0 -1039.1643756908593) / Math.Sqrt((26407.78064802967 / (43)) + (273.6402759492868 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.35408489479215316 = (767.4292650981585 - 1039.1643756908593) / 767.4292650981585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToString(Location).JsonNet.html>) 32.66 μs 35.53 μs 1.09 0.01 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(Location).SystemTextJsonSourceGen.html>) 28.67 μs 31.69 μs 1.11 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(Location).SystemTextJsonReflection.html>) 37.94 μs 41.49 μs 1.09 0.01 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<Location>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.52631792458571 > 34.465194221026785. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.84559030564165 (T) = (0 -35276.6882493432) / Math.Sqrt((274963.6758183933 / (44)) + (67446.70537041157 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06859709693172 = (33012.15055761776 - 35276.6882493432) / 33012.15055761776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.689846828231286 > 30.096693757147296. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.95878680663913 (T) = (0 -31564.5231504563) / Math.Sqrt((94065.289258743 / (44)) + (68137.96631250245 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1086451386088186 = (28471.25924357093 - 31564.5231504563) / 28471.25924357093 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.49313303023846 > 39.63708554725269. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.001414693643866 (T) = (0 -41312.97111225873) / Math.Sqrt((286676.5422256363 / (44)) + (73237.49802186868 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10889716373792474 = (37255.9083594361 - 41312.97111225873) / 37255.9083594361 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Convert_uint_float - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float.html>) 285.66 μs 324.39 μs 1.14 0.07 False
[Widen_sbyte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte.html>) 587.59 μs 702.75 μs 1.20 0.06 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorConvert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorConvert.Convert_uint_float ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 324.3925159013606 > 292.1699647334917. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.397747995207306 (T) = (0 -325839.49432953954) / Math.Sqrt((482831985.22406214 / (43)) + (4266967.647053949 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1482410873614179 = (283772.71804329625 - 325839.49432953954) / 283772.71804329625 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorConvert.Widen_sbyte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 702.7483702898552 > 618.1633190073005. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.481785779141173 (T) = (0 -704860.7659271224) / Math.Sqrt((886527958.1969807 / (44)) + (22882209.98498111 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17438349230142042 = (600196.4184167968 - 704860.7659271224) / 600196.4184167968 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_StreamWriter.WriteFormat.html>) 2.05 μs 2.63 μs 1.28 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_StreamWriter*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_StreamWriter.WriteFormat ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.633850983678434 > 2.1646969949471244. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.987647498112793 (T) = (0 -2627.400948744588) / Math.Sqrt((17854.670343921534 / (44)) + (1517.4372862366602 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27247848984784456 = (2064.7900689140583 - 2627.400948744588) / 2064.7900689140583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.TryAddDefaultSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddDefaultSize(String).ConcurrentDictionary(Count%3a%20512).html>) 553.49 μs 619.87 μs 1.12 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<String>.ConcurrentDictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 619.8677886332418 > 592.0679575837054. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.475433234442633 (T) = (0 -603821.9827330342) / Math.Sqrt((245091977.56543496 / (44)) + (150068471.2609004 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07349809923729816 = (562480.7190269263 - 603821.9827330342) / 562480.7190269263 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(Location).XmlSerializer.html>) 174.16 μs 186.62 μs 1.07 0.01 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(Location).DataContractSerializer.html>) 168.05 μs 182.31 μs 1.08 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<Location>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 186.6223955109127 > 183.3040763020833. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.67502887633323 (T) = (0 -187672.5635404866) / Math.Sqrt((2325303.5652691303 / (43)) + (703267.1632378793 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.07787903068084946 = (174112.82546422855 - 187672.5635404866) / 174112.82546422855 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_FromStream<Location>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 182.30507687915286 > 177.79027384593047. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.65325053601894 (T) = (0 -182985.62833948346) / Math.Sqrt((3066000.3190137427 / (43)) + (1185737.5121956482 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.08921469951774902 = (167997.7587710674 - 182985.62833948346) / 167997.7587710674 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ServiceProvider - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider.html>) 5.12 μs 5.49 μs 1.07 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.4870861638287804 > 5.39133920384505. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.56417449756765 (T) = (0 -5580.565374311204) / Math.Sqrt((7747.580581498648 / (44)) + (2648.2247342403803 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09010564538864255 = (5119.288573468153 - 5580.565374311204) / 5119.288573468153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Buffers.Tests.NonStandardArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[RentNoReturn - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.NonStandardArrayPoolTests(Byte).RentNoReturn(RentalSize%3a%2064%2c%20UseSharedPool%3a%20True).html>) 207.75 ns 246.72 ns 1.19 0.11 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.NonStandardArrayPoolTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.NonStandardArrayPoolTests<Byte>.RentNoReturn(RentalSize: 64, UseSharedPool: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 246.72040013333333 > 220.62849429335665. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.727169942898552 (T) = (0 -229.77036354322124) / Math.Sqrt((10.492279313457757 / (43)) + (28.255904839556944 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.08789923983193561 = (211.20555574495793 - 229.77036354322124) / 211.20555574495793 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StartsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).StartsWith(Size%3a%20512).html>) 753.26 ns 859.03 ns 1.14 0.07 False
[SequenceCompareToDifferent - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).SequenceCompareToDifferent(Size%3a%20512).html>) 138.00 ns 197.52 ns 1.43 0.16 False
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).ToArray(Size%3a%20512).html>) 443.04 ns 571.40 ns 1.29 0.07 False
[Fill - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).Fill(Size%3a%20512).html>) 51.69 ns 66.44 ns 1.29 0.13 False
[BinarySearch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).BinarySearch(Size%3a%20512).html>) 324.91 ns 379.67 ns 1.17 0.08 False
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).SequenceEqual(Size%3a%20512).html>) 1.30 μs 1.40 μs 1.07 0.06 False
[GetPinnableReference - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).GetPinnableReference(Size%3a%20512).html>) 44.44 ns 47.79 ns 1.08 0.04 False
[EndsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).EndsWith(Size%3a%20512).html>) 755.24 ns 823.61 ns 1.09 0.07 False
[IndexOfAnyFiveValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).IndexOfAnyFiveValues(Size%3a%20512).html>) 6.54 μs 23.13 μs 3.53 0.10 False
[IndexOfAnyFourValues - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Byte).IndexOfAnyFourValues(Size%3a%20512).html>) 15.95 μs 19.00 μs 1.19 0.12 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Byte>.StartsWith(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 859.0273018930747 > 791.5085053571696. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.4484378255745 (T) = (0 -842.9928489698601) / Math.Sqrt((178.3386273008288 / (44)) + (39.240725134593795 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11416655707575145 = (756.6129530780249 - 842.9928489698601) / 756.6129530780249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 197.51735605171805 > 145.18541751617462. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.674091469774844 (T) = (0 -197.8748492198092) / Math.Sqrt((22.20840296573487 / (44)) + (21.782020732451986 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40360075768721737 = (140.97659048421818 - 197.8748492198092) / 140.97659048421818 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 571.3959200930402 > 461.7676539044059. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.722294692033616 (T) = (0 -572.8466049470336) / Math.Sqrt((7388.15243333965 / (44)) + (40.00163881916355 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32356138671493473 = (432.806978729436 - 572.8466049470336) / 432.806978729436 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.Fill(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.43774483124942 > 55.27496340593713. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.71272583558387 (T) = (0 -66.85724077894773) / Math.Sqrt((1.0345659521024437 / (44)) + (2.6186499814013566 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2783971926374617 = (52.29770619334241 - 66.85724077894773) / 52.29770619334241 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.BinarySearch(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 379.6662422174532 > 340.92476412310515. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.26180165968605 (T) = (0 -376.8552466439295) / Math.Sqrt((35.62604402113967 / (43)) + (4.97333561362919 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15236282107060803 = (327.0282933059315 - 376.8552466439295) / 327.0282933059315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.SequenceEqual(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3966105164544715 > 1.3592346172693073. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.343552297876125 (T) = (0 -1407.4028799920075) / Math.Sqrt((640.2045931369986 / (44)) + (2904.456334638727 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08141414251355254 = (1301.446711914413 - 1407.4028799920075) / 1301.446711914413 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.GetPinnableReference(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.790001166831026 > 47.12196555165014. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/10/2022 12:36:40 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -2.830944139859714 (T) = (0 -48.342715467005924) / Math.Sqrt((9.970050447563457 / (46)) + (11.077835079605673 / (12))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (12) - 2, .025) and -0.06669161126564868 = (45.32023591115189 - 48.342715467005924) / 45.32023591115189 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.EndsWith(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 823.6104615321387 > 791.0472652081115. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.929811789361327 (T) = (0 -821.0743017090828) / Math.Sqrt((563.406188127159 / (44)) + (13.767755317411954 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08269783401406146 = (758.3596049739757 - 821.0743017090828) / 758.3596049739757 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.134120297619052 > 6.669292620547413. IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.836997583759405 (T) = (0 -23421.685115402706) / Math.Sqrt((205491.5541761097 / (44)) + (1983537.2805473488 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.6247392382465002 = (6461.619326507237 - 23421.685115402706) / 6461.619326507237 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.00076579336114 > 16.75980007167311. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.728414819481 (T) = (0 -19033.268739405798) / Math.Sqrt((135851.42749386048 / (44)) + (567.2714103916708 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18738333514999403 = (16029.59059300041 - 19033.268739405798) / 16029.59059300041 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Guids

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 174.09 ms 205.28 ms 1.18 0.03 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 185.93 ms 220.45 ms 1.19 0.03 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 189.48 ms 223.46 ms 1.18 0.04 False
[WriteGuids - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 169.46 ms 200.29 ms 1.18 0.03 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Guids*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 205.27671407142856 > 184.24277499750002. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.817232545802675 (T) = (0 -203971653.6696215) / Math.Sqrt((5822554931588.932 / (44)) + (2990495495390.2466 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1636881628205635 = (175280337.2814519 - 203971653.6696215) / 175280337.2814519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 220.44793353333333 > 196.311712425. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.58905178937469 (T) = (0 -218693654.90622705) / Math.Sqrt((2790209680329.846 / (44)) + (4415994810315.683 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16428989777927608 = (187834366.10019147 - 218693654.90622705) / 187834366.10019147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 223.45860046666667 > 200.75463663230772. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.88412510787619 (T) = (0 -221663923.75995117) / Math.Sqrt((14354805141016.799 / (44)) + (3506641374628.981 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15560223879492907 = (191816800.20895773 - 221663923.75995117) / 191816800.20895773 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Guids.WriteGuids(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.2888666 > 179.45276180730772. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.65727240598133 (T) = (0 -201356380.38793647) / Math.Sqrt((10810129658304.951 / (44)) + (7769988982823.0625 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16513875930796534 = (172817511.02978685 - 201356380.38793647) / 172817511.02978685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(SimpleStructWithProperties).DataContractSerializer.html>) 68.42 μs 73.55 μs 1.08 0.01 False
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlFromStream(SimpleStructWithProperties).XmlSerializer.html>) 128.50 μs 135.24 μs 1.05 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.55473553178042 > 72.211906944028. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.336446730312694 (T) = (0 -73270.3084410003) / Math.Sqrt((644827.3157027658 / (44)) + (57656.000172614986 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06807740728241636 = (68600.18566203647 - 73270.3084410003) / 68600.18566203647 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_FromStream<SimpleStructWithProperties>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.23629666149066 > 134.95584773927567. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.2248864000392 (T) = (0 -136075.8550154449) / Math.Sqrt((907301.78911208 / (44)) + (910598.9303126641 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.060508557505004466 = (128311.88777541077 - 136075.8550154449) / 128311.88777541077 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToHexString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Perf_Convert.ToHexString.html>) 4.11 μs 5.29 μs 1.29 0.07 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Perf_Convert.ToDateTime_String(value%3a%20%22February%2026%2c%202009%22).html>) 7.55 μs 8.59 μs 1.14 0.03 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Perf_Convert.ToDateTime_String(value%3a%20%22Fri%2c%2027%20Feb%202009%2003%3a11%3a21%20GMT%22).html>) 34.21 μs 38.93 μs 1.14 0.05 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Perf_Convert.ToDateTime_String(value%3a%20%2212%2f12%2f1999%2011%3a59%3a59%20PM%22).html>) 11.79 μs 13.10 μs 1.11 0.01 False
[ToDateTime_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Perf_Convert.ToDateTime_String(value%3a%20%2212%2f12%2f1999%22).html>) 5.76 μs 6.25 μs 1.09 0.02 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Perf_Convert.ToHexString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.287674743375871 > 4.287183355014564. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.427124797052212 (T) = (0 -5240.4319211823085) / Math.Sqrt((6153.936359293584 / (44)) + (21324.62134257177 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2851775848040447 = (4077.5936206367423 - 5240.4319211823085) / 4077.5936206367423 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Perf_Convert.ToDateTime_String(value: "February 26, 2009") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.586220970247785 > 8.15283815412444. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.90761006437253 (T) = (0 -8569.544111279978) / Math.Sqrt((11274.292107564801 / (44)) + (6653.304669700979 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10925771766842358 = (7725.476212410328 - 8569.544111279978) / 7725.476212410328 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Perf_Convert.ToDateTime_String(value: "Fri, 27 Feb 2009 03:11:21 GMT") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.93397916528432 > 36.34470119804336. IsChangePoint: Marked as a change because one of 10/7/2022 12:26:39 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.007891209382965 (T) = (0 -38648.301062886385) / Math.Sqrt((2552784.7682788786 / (44)) + (124264.58767903355 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1277950471703307 = (34268.90476231125 - 38648.301062886385) / 34268.90476231125 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999 11:59:59 PM") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.099032446121067 > 12.521061352753687. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.07782740783155 (T) = (0 -13119.318539108268) / Math.Sqrt((14380.323461286838 / (44)) + (9935.040900796095 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1033645931620206 = (11890.284154860312 - 13119.318539108268) / 11890.284154860312 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Perf_Convert.ToDateTime_String(value: "12/12/1999") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.253505725210041 > 6.157446802948223. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.22165000700373 (T) = (0 -6342.9516619935) / Math.Sqrt((12770.198025484486 / (44)) + (8063.586713035376 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.08684843478515418 = (5836.095870393705 - 6342.9516619935) / 5836.095870393705 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromString(Mode%3a%20Reflection).html>) 289.81 μs 310.95 μs 1.07 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 240.84 μs 264.97 μs 1.10 0.04 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 328.54 μs 355.12 μs 1.08 0.04 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 240.56 μs 257.04 μs 1.07 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(HashSet(String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 326.77 μs 352.03 μs 1.08 0.01 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 310.9462428385417 > 303.9059532810274. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.063819923753883 (T) = (0 -312861.0602187521) / Math.Sqrt((18502055.170894973 / (43)) + (10229311.007476667 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.07643108349067633 = (290646.62384534534 - 312861.0602187521) / 290646.62384534534 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 264.9683343055556 > 251.70695913461543. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.700264115178285 (T) = (0 -259897.73529858512) / Math.Sqrt((9325543.31716962 / (44)) + (11156295.513161013 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08443807121338852 = (239661.20537227447 - 259897.73529858512) / 239661.20537227447 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 355.1230105587121 > 343.592921484375. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.088278848503302 (T) = (0 -361303.5484058069) / Math.Sqrt((15866781.487807496 / (44)) + (172646258.01816258 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10643994138634927 = (326546.01021823206 - 361303.5484058069) / 326546.01021823206 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 257.037637363388 > 250.922416756287. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.074763667022065 (T) = (0 -257343.44247011977) / Math.Sqrt((11271277.411945295 / (44)) + (9665857.021256024 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07566188726128087 = (239241.94537126928 - 257343.44247011977) / 239241.94537126928 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<HashSet<String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 352.03458263888893 > 341.15454220982144. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.532695541781845 (T) = (0 -355666.3731662814) / Math.Sqrt((18026049.279304054 / (44)) + (15076887.070001751 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09336119074460285 = (325296.3212678738 - 355666.3731662814) / 325296.3212678738 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).LessThanOrEqualAllBenchmark.html>) 75.04 ns 272.26 ns 3.63 0.10 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GreaterThanOrEqualAllBenchmark.html>) 75.41 ns 273.96 ns 3.63 0.06 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).LessThanAnyBenchmark.html>) 180.48 ns 223.57 ns 1.24 0.21 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).OnesComplementOperatorBenchmark.html>) 103.46 ns 153.44 ns 1.48 0.12 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).LessThanOrEqualAnyBenchmark.html>) 169.23 ns 243.74 ns 1.44 0.23 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).ConditionalSelectBenchmark.html>) 179.98 ns 417.89 ns 2.32 0.04 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).AndNotBenchmark.html>) 133.49 ns 263.84 ns 1.98 0.08 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GetHashCodeBenchmark.html>) 399.02 ns 461.59 ns 1.16 0.13 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).LessThanAllBenchmark.html>) 73.25 ns 272.25 ns 3.72 0.02 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).OnesComplementBenchmark.html>) 102.19 ns 152.20 ns 1.49 0.14 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).SumBenchmark.html>) 46.04 ns 77.28 ns 1.68 0.15 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GreaterThanAllBenchmark.html>) 76.84 ns 272.03 ns 3.54 0.12 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GreaterThanOrEqualAnyBenchmark.html>) 184.04 ns 229.53 ns 1.25 0.20 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).AllBitsSetBenchmark.html>) 44.72 ns 87.39 ns 1.95 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).EqualsAnyBenchmark.html>) 165.59 ns 223.49 ns 1.35 0.21 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Double).GreaterThanAnyBenchmark.html>) 171.82 ns 227.59 ns 1.32 0.17 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.2606345960056 > 80.0719357962466. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -177.59362178796613 (T) = (0 -274.05168730354046) / Math.Sqrt((3.747923163563452 / (44)) + (17.197745866737684 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5611916533230845 = (76.95505156197149 - 274.05168730354046) / 76.95505156197149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 273.9592129130307 > 80.31571898482484. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -256.5285735706937 (T) = (0 -273.60271142075254) / Math.Sqrt((15.961962462020358 / (44)) + (3.325595118069335 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5311046661384817 = (77.4836028069247 - 273.60271142075254) / 77.4836028069247 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 223.5715143907718 > 176.07013819201845. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.310745750974796 (T) = (0 -226.20433798445234) / Math.Sqrt((32.97920161321745 / (44)) + (8.378527095087684 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3388593789581373 = (168.9530219077064 - 226.20433798445234) / 168.9530219077064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.43954209132676 > 111.49008293602323. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.7403433521222365 (T) = (0 -155.67489831527092) / Math.Sqrt((970.3096510842802 / (44)) + (45.21909822985737 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2767774207320116 = (121.92798508765782 - 155.67489831527092) / 121.92798508765782 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 243.74199584506334 > 177.73740404958812. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.76553754047956 (T) = (0 -230.93024997617943) / Math.Sqrt((47.497379628989606 / (44)) + (35.26676801136663 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34188068463861304 = (172.09447353985286 - 230.93024997617943) / 172.09447353985286 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.8875075615796 > 192.06370412940745. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.76485945142676 (T) = (0 -424.3247987936495) / Math.Sqrt((953.0622844490117 / (44)) + (134.83168600988583 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1361152724972756 = (198.6432119356474 - 424.3247987936495) / 198.6432119356474 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 263.84186257477467 > 141.69094510922486. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.9817299264898 (T) = (0 -265.80843939126333) / Math.Sqrt((927.6350712740261 / (44)) + (18.21039154556205 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7422253849235894 = (152.56834258727162 - 265.80843939126333) / 152.56834258727162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 461.59498746074894 > 424.2003408063977. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.1381669408958635 (T) = (0 -468.38029326447) / Math.Sqrt((899.3236691701156 / (44)) + (996.690617208173 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1391539151779054 = (411.16506472377927 - 468.38029326447) / 411.16506472377927 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.249326734678 > 77.52695123628945. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -269.3444744560375 (T) = (0 -271.29392356449154) / Math.Sqrt((1.0300281726446892 / (44)) + (7.63876851042895 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.6309744981180088 = (74.71655989462538 - 271.29392356449154) / 74.71655989462538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.19779152564263 > 107.98011612081751. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.035481192008186 (T) = (0 -157.14254925668453) / Math.Sqrt((894.6103125083197 / (44)) + (69.75120561804466 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2882889970148231 = (121.97771588580636 - 157.14254925668453) / 121.97771588580636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 77.28411708238525 > 47.9359331450533. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.709495003964632 (T) = (0 -64.02459524942508) / Math.Sqrt((6.709861012039585 / (44)) + (38.27062233720719 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3794097408584976 = (46.414486829401646 - 64.02459524942508) / 46.414486829401646 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.0349588371203 > 78.93116883880968. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -105.11638164653617 (T) = (0 -272.44579228672416) / Math.Sqrt((4.03621341270585 / (43)) + (51.32349073867739 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -2.6153532097900203 = (75.35800141158208 - 272.44579228672416) / 75.35800141158208 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 229.5324173952459 > 178.67810493643836. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.232850115119085 (T) = (0 -228.54102358115915) / Math.Sqrt((57.827352855900486 / (44)) + (2.17494811346437 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3142100757320032 = (173.89991737345633 - 228.54102358115915) / 173.89991737345633 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.3904800912087 > 47.70987568367078. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.677053137858165 (T) = (0 -88.51030143232462) / Math.Sqrt((910.4241203292958 / (44)) + (2.5336350899551645 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.41432254912302185 = (62.58141149429255 - 88.51030143232462) / 62.58141149429255 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 223.49420757835458 > 175.7620605303346. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.98514883751204 (T) = (0 -226.78329423958118) / Math.Sqrt((25.53596734130222 / (43)) + (89.7964019441523 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.3452299522125195 = (168.58329229630024 - 226.78329423958118) / 168.58329229630024 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 227.59195904085104 > 184.45775678731133. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -77.34498427032992 (T) = (0 -225.9359338423781) / Math.Sqrt((19.905803641836684 / (44)) + (1.5006286550982726 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3412806177129197 = (168.4479227230108 - 225.9359338423781) / 168.4479227230108 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).AbsBenchmark.html>) 74.76 ns 130.28 ns 1.74 0.11 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).GreaterThanAnyBenchmark.html>) 79.37 ns 227.44 ns 2.87 0.12 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).OneBenchmark.html>) 43.05 ns 83.91 ns 1.95 0.09 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanOrEqualBenchmark.html>) 114.46 ns 128.58 ns 1.12 0.10 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).OnesComplementBenchmark.html>) 67.83 ns 156.42 ns 2.31 0.09 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanOrEqualAnyBenchmark.html>) 57.35 ns 185.53 ns 3.24 0.13 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).MultiplyOperatorBenchmark.html>) 108.03 ns 118.57 ns 1.10 0.08 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).NegateBenchmark.html>) 103.90 ns 133.74 ns 1.29 0.08 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).GreaterThanAllBenchmark.html>) 55.27 ns 254.55 ns 4.61 0.06 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).AddBenchmark.html>) 103.77 ns 136.72 ns 1.32 0.10 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).SubtractBenchmark.html>) 102.85 ns 144.32 ns 1.40 0.09 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).UnaryNegateOperatorBenchmark.html>) 102.13 ns 136.80 ns 1.34 0.09 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanAllBenchmark.html>) 80.34 ns 319.68 ns 3.98 0.09 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).OnesComplementOperatorBenchmark.html>) 66.55 ns 158.29 ns 2.38 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanOrEqualAllBenchmark.html>) 82.03 ns 337.93 ns 4.12 0.11 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanBenchmark.html>) 111.27 ns 123.64 ns 1.11 0.09 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).DivisionOperatorBenchmark.html>) 104.48 ns 118.77 ns 1.14 0.09 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).DivideBenchmark.html>) 106.69 ns 137.20 ns 1.29 0.08 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).LessThanAnyBenchmark.html>) 54.50 ns 185.07 ns 3.40 0.10 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).GreaterThanOrEqualAllBenchmark.html>) 56.61 ns 258.21 ns 4.56 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).EqualsAnyBenchmark.html>) 79.02 ns 226.85 ns 2.87 0.08 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).GreaterThanOrEqualAnyBenchmark.html>) 89.12 ns 248.28 ns 2.79 0.09 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).SubtractionOperatorBenchmark.html>) 106.04 ns 116.72 ns 1.10 0.06 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).MultiplyBenchmark.html>) 103.50 ns 136.73 ns 1.32 0.06 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).AddOperatorBenchmark.html>) 102.80 ns 117.79 ns 1.15 0.05 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Double).AndNotBenchmark.html>) 168.94 ns 269.64 ns 1.60 0.08 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Double>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.2781684460469 > 78.50761230070412. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -127.02790356753127 (T) = (0 -130.60323723905333) / Math.Sqrt((2.380063276654726 / (44)) + (2.093314139384156 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7482709773158692 = (74.70423002706895 - 130.60323723905333) / 74.70423002706895 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 227.43559408225903 > 83.59388020545008. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.58033647820227 (T) = (0 -231.5582617890595) / Math.Sqrt((3.6894523606275356 / (44)) + (81.46011912518115 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.8979529127719013 = (79.90408014172088 - 231.5582617890595) / 79.90408014172088 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 83.90869133840032 > 44.751987168480966. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -92.63300106638225 (T) = (0 -81.05114656858969) / Math.Sqrt((1.138105276712094 / (44)) + (2.1790414439294543 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8968098481774416 = (42.730243438196005 - 81.05114656858969) / 42.730243438196005 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 128.58358228701215 > 121.56793265246861. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.46686182397857 (T) = (0 -127.72355713199367) / Math.Sqrt((7.008902701837228 / (44)) + (6.96679144447259 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11336566763134091 = (114.71842616067237 - 127.72355713199367) / 114.71842616067237 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 156.41651034914122 > 70.61286405143372. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -153.49240738518026 (T) = (0 -158.0884933665981) / Math.Sqrt((2.8610291310151394 / (44)) + (4.23364261685877 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3373253920346446 = (67.63649336346013 - 158.0884933665981) / 67.63649336346013 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 185.52935823122638 > 59.35767564773188. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -164.13368031432154 (T) = (0 -186.8295343511358) / Math.Sqrt((9.353259277301119 / (44)) + (6.158612057080421 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.262666323088101 = (57.26283838130967 - 186.8295343511358) / 57.26283838130967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 118.56895991130871 > 109.31688890797143. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.777074392940234 (T) = (0 -120.53949141018856) / Math.Sqrt((14.661352194055649 / (44)) + (30.449275477260795 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15934190873627424 = (103.9723402577425 - 120.53949141018856) / 103.9723402577425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 133.7440610113752 > 107.01854137375423. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.665776105363424 (T) = (0 -135.74593952483977) / Math.Sqrt((2.752265565874287 / (44)) + (18.11209201452942 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32675836590834495 = (102.31398799728191 - 135.74593952483977) / 102.31398799728191 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 254.54651981968118 > 57.693461821442874. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -142.6799482047805 (T) = (0 -255.61665190600766) / Math.Sqrt((1.1166950338396762 / (44)) + (29.214140050473986 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.6304053559617993 = (55.2039470101452 - 255.61665190600766) / 55.2039470101452 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.7178874257893 > 108.22878342685246. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -74.43473498910822 (T) = (0 -136.64491419311773) / Math.Sqrt((2.625303475582743 / (44)) + (2.1546968661637953 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3255923037562573 = (103.08215716545322 - 136.64491419311773) / 103.08215716545322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.3177529232736 > 109.62102431866249. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.283078465172185 (T) = (0 -138.3462001042265) / Math.Sqrt((4.505388951794454 / (44)) + (27.196799430675128 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3385680471834453 = (103.35387909141292 - 138.3462001042265) / 103.35387909141292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.7979608576551 > 107.3547660038949. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.99315442785354 (T) = (0 -135.6247989070984) / Math.Sqrt((2.246843285246642 / (44)) + (25.855294554374666 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32538022718155657 = (102.32897407524091 - 135.6247989070984) / 102.32897407524091 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 319.6750721748992 > 83.50417455320373. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -203.0293276627871 (T) = (0 -321.8659022837578) / Math.Sqrt((1.4709822281323854 / (44)) + (19.48161785514712 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -3.0483708605542272 = (79.50504372509339 - 321.8659022837578) / 79.50504372509339 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.29420194908235 > 70.37915777277055. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -138.01564431040154 (T) = (0 -158.53202727557922) / Math.Sqrt((1.1451740809110622 / (44)) + (6.174132423508609 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3580856259689993 = (67.2291224414017 - 158.53202727557922) / 67.2291224414017 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 337.92629582038353 > 87.14510693249511. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -148.9050590258725 (T) = (0 -326.3965673332386) / Math.Sqrt((14.358035505850257 / (44)) + (35.031782717717036 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.910864719848778 = (83.45892550992134 - 326.3965673332386) / 83.45892550992134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.64165037312605 > 116.90986767523808. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 9/29/2022 5:24:34 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.87362276254406 (T) = (0 -123.98154816741202) / Math.Sqrt((3.7381633644844325 / (44)) + (1.1172259108853242 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1184105476056907 = (110.855131358367 - 123.98154816741202) / 110.855131358367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 118.77428941751289 > 110.20954136657399. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.458828036136122 (T) = (0 -120.09791983002849) / Math.Sqrt((11.773812461204006 / (44)) + (2.563195810809771 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1485563722553078 = (104.56423622830454 - 120.09791983002849) / 104.56423622830454 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 137.20207773361548 > 110.53284621440497. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.0141012621284 (T) = (0 -138.53247699808884) / Math.Sqrt((2.725187651618439 / (44)) + (3.173463767868962 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3186679963480087 = (105.05485640187543 - 138.53247699808884) / 105.05485640187543 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 185.0656345128713 > 57.54036452397587. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -125.4903060434362 (T) = (0 -183.85706671214936) / Math.Sqrt((2.6069441626586345 / (44)) + (14.738660476555003 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.296745764145219 = (55.76925849476897 - 183.85706671214936) / 55.76925849476897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 258.21206079003235 > 60.311048544816245. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -170.40131378340638 (T) = (0 -258.8083180014701) / Math.Sqrt((25.52466260781331 / (44)) + (12.104475095281344 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.4530059234330883 = (58.119913256692755 - 258.8083180014701) / 58.119913256692755 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 226.84640034066703 > 83.66304663620882. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -83.74262852839104 (T) = (0 -230.04872207994353) / Math.Sqrt((1.295898315735103 / (43)) + (47.94929248805409 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -1.8893199170211674 = (79.62037042859528 - 230.04872207994353) / 79.62037042859528 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 248.28353973809988 > 86.06495917019143. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.43189641717474 (T) = (0 -234.3301795620278) / Math.Sqrt((4.634326792628519 / (44)) + (58.71960527234138 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.8201856636170215 = (83.0903378402003 - 234.3301795620278) / 83.0903378402003 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 116.72052939946548 > 108.73852550981753. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.72668949109018 (T) = (0 -119.44092266545485) / Math.Sqrt((3.9641080943882963 / (44)) + (8.094006169652577 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1508256848179437 = (103.78715407655326 - 119.44092266545485) / 103.78715407655326 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 136.7346210593434 > 109.12391568321657. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -100.71435765901205 (T) = (0 -135.7343564153498) / Math.Sqrt((3.088396944615601 / (44)) + (0.4958966664672792 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3130646554208164 = (103.37218038349306 - 135.7343564153498) / 103.37218038349306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 117.78767170000032 > 107.88730021091453. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.29016401458363 (T) = (0 -118.43239725202741) / Math.Sqrt((3.995581729685244 / (44)) + (0.37719920444640404 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1459282892546489 = (103.35061832626535 - 118.43239725202741) / 103.35061832626535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Double>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 269.6357096678073 > 178.57598184560763. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -130.329509413825 (T) = (0 -269.71981081147084) / Math.Sqrt((17.50516787974541 / (44)) + (2.5957319225573303 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5750420915132278 = (171.24609701848442 - 269.71981081147084) / 171.24609701848442 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Span.IndexerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[KnownSizeArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.KnownSizeArray(length%3a%201024).html>) 12.92 μs 18.22 μs 1.41 0.08 False
[KnownSizeCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.KnownSizeCtor(length%3a%201024).html>) 13.60 μs 18.18 μs 1.34 0.08 False
[Fixed1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Fixed1(length%3a%201024).html>) 9.36 μs 11.05 μs 1.18 0.07 False
[SameIndex2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.SameIndex2(length%3a%201024).html>) 22.09 μs 29.37 μs 1.33 0.08 False
[SameIndex1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.SameIndex1(length%3a%201024).html>) 18.95 μs 25.84 μs 1.36 0.12 False
[WriteViaIndexer2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.WriteViaIndexer2(length%3a%201024).html>) 15.36 μs 23.71 μs 1.54 0.07 False
[Fixed2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Fixed2(length%3a%201024).html>) 9.36 μs 11.06 μs 1.18 0.07 False
[Indexer5 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer5(length%3a%201024).html>) 15.43 μs 20.45 μs 1.33 0.06 False
[Indexer3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer3(length%3a%201024).html>) 12.92 μs 18.20 μs 1.41 0.09 False
[KnownSizeCtor2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.KnownSizeCtor2(length%3a%201024).html>) 12.98 μs 18.26 μs 1.41 0.12 False
[CoveredIndex3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex3(length%3a%201024).html>) 30.89 μs 41.74 μs 1.35 0.09 False
[Indexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer1(length%3a%201024).html>) 11.50 μs 14.97 μs 1.30 0.09 False
[Ref - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Ref(length%3a%201024).html>) 9.36 μs 11.06 μs 1.18 0.04 False
[ReadOnlyIndexer2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.ReadOnlyIndexer2(length%3a%201024).html>) 12.93 μs 18.20 μs 1.41 0.11 False
[WriteViaIndexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.WriteViaIndexer1(length%3a%201024).html>) 20.24 μs 25.46 μs 1.26 0.13 False
[CoveredIndex2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex2(length%3a%201024).html>) 22.23 μs 29.15 μs 1.31 0.10 False
[Indexer2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.Indexer2(length%3a%201024).html>) 12.92 μs 18.28 μs 1.41 0.09 False
[CoveredIndex1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.CoveredIndex1(length%3a%201024).html>) 21.98 μs 29.01 μs 1.32 0.09 False
[ReadOnlyIndexer1 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Span.IndexerBench.ReadOnlyIndexer1(length%3a%201024).html>) 11.49 μs 15.73 μs 1.37 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Span.IndexerBench.KnownSizeArray(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.215572402692114 > 13.570580965506716. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.2475754857398 (T) = (0 -18260.406151590487) / Math.Sqrt((149568.3152677711 / (44)) + (40488.289022441335 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4036911702056751 = (13008.848769003007 - 18260.406151590487) / 13008.848769003007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.KnownSizeCtor(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.18449259074259 > 14.022352696443344. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.74961910808341 (T) = (0 -18192.52590159064) / Math.Sqrt((148780.74400135787 / (44)) + (422.0441314421177 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39206512187155884 = (13068.731926227518 - 18192.52590159064) / 13068.731926227518 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Fixed1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.054925062783207 > 9.82732962171845. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.57081866342013 (T) = (0 -11063.331658436904) / Math.Sqrt((81582.63411263713 / (44)) + (56.26311260314816 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1714788391273123 = (9443.90226175872 - 11063.331658436904) / 9443.90226175872 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.SameIndex2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.371366016604007 > 23.198894253219926. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.571531494818956 (T) = (0 -29479.90863476349) / Math.Sqrt((422892.8625221157 / (44)) + (462610.86756342987 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32045751898316227 = (22325.52597183507 - 29479.90863476349) / 22325.52597183507 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.SameIndex1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.838093163631346 > 19.920014798490747. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -82.63804861584627 (T) = (0 -25859.8701279576) / Math.Sqrt((290461.4345647627 / (44)) + (530.1363853434505 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35196175934603935 = (19127.663892258566 - 25859.8701279576) / 19127.663892258566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.WriteViaIndexer2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.70976859509358 > 16.127580772164126. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -293.2921663723075 (T) = (0 -23709.091075887012) / Math.Sqrt((34672.338401109286 / (44)) + (113.01023821868641 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5358928326966361 = (15436.683192446373 - 23709.091075887012) / 15436.683192446373 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Fixed2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.055290119834034 > 9.83215373401295. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.056943438451643 (T) = (0 -11090.838777967168) / Math.Sqrt((265093.46560183377 / (44)) + (5540.12785226998 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16905507434316527 = (9487.01136616559 - 11090.838777967168) / 9487.01136616559 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer5(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.453929612900208 > 15.952944474474402. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -85.4583697867751 (T) = (0 -20476.861062779306) / Math.Sqrt((161804.97120938278 / (44)) + (1211.4945707959355 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34380761282585737 = (15237.94095772315 - 20476.861062779306) / 15237.94095772315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer3(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.200857199757653 > 13.573343592067108. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -69.86101646962099 (T) = (0 -18209.001144679252) / Math.Sqrt((233159.7586320071 / (44)) + (1271.4947867145006 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.391809698012519 = (13082.967571415404 - 18209.001144679252) / 13082.967571415404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.KnownSizeCtor2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.26361589668616 > 13.630340965838185. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.25897635456863 (T) = (0 -18256.534276041275) / Math.Sqrt((121387.76086910073 / (44)) + (100.63104060787302 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3891898149115807 = (13141.85727542443 - 18256.534276041275) / 13141.85727542443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex3(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.743662381735916 > 32.35597158855202. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.67933782486306 (T) = (0 -42947.50005243273) / Math.Sqrt((193161.2128752699 / (44)) + (13439358.910955178 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38914480103331006 = (30916.50346348804 - 42947.50005243273) / 30916.50346348804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.969849576135637 > 12.074263894266565. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.82018947885511 (T) = (0 -14979.926780313805) / Math.Sqrt((55012.074269045006 / (44)) + (69.8935715577982 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2969558370366601 = (11550.066974169744 - 14979.926780313805) / 11550.066974169744 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Ref(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.056483401300234 > 9.824600223709282. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.998798653520005 (T) = (0 -11072.46839069473) / Math.Sqrt((131634.78209550426 / (44)) + (665.950668637986 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17548228177950428 = (9419.511091169039 - 11072.46839069473) / 9419.511091169039 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.ReadOnlyIndexer2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.200913699158196 > 13.573178340151204. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -161.60116499850554 (T) = (0 -18240.143680924368) / Math.Sqrt((26296.5624454017 / (44)) + (7077.187981175833 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40792241381341265 = (12955.361390632477 - 18240.143680924368) / 12955.361390632477 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.WriteViaIndexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.459358145626805 > 21.25170877787273. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.898805963267971 (T) = (0 -25559.84558860992) / Math.Sqrt((6700023.94263752 / (44)) + (142541.92173978325 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2303778990592313 = (20773.979773330964 - 25559.84558860992) / 20773.979773330964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.150140010883085 > 23.348614171392178. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.125911384000794 (T) = (0 -30404.942668927684) / Math.Sqrt((357983.84880837746 / (44)) + (25246604.51279962 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3550448224745312 = (22438.32983576391 - 30404.942668927684) / 22438.32983576391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.Indexer2(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.28027913251366 > 13.569078283790184. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/24/2022 11:57:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.4404896321225 (T) = (0 -18216.07145019672) / Math.Sqrt((186645.64757991087 / (44)) + (734.1542804466043 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39990837759052883 = (13012.331193809667 - 18216.07145019672) / 13012.331193809667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.CoveredIndex1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.010928372278276 > 23.079954612555053. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.219868235197755 (T) = (0 -29431.008129780945) / Math.Sqrt((322904.7895516554 / (44)) + (764306.1370159417 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3281555474815073 = (22159.308211744476 - 29431.008129780945) / 22159.308211744476 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Span.IndexerBench.ReadOnlyIndexer1(length: 1024) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.730036722350231 > 12.069252122558954. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.23246862148841 (T) = (0 -15210.4292033828) / Math.Sqrt((214787.60486347644 / (44)) + (376274.3185856337 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3106980261990251 = (11604.831089501578 - 15210.4292033828) / 11604.831089501578 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark.html>) 100.28 ns 118.81 ns 1.18 0.09 False
[CreateScaleFromScalarXYWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark.html>) 68.80 ns 96.71 ns 1.41 0.10 False
[CreateScaleFromScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark.html>) 46.11 ns 70.43 ns 1.53 0.16 False
[InvertBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark.html>) 109.70 ns 122.30 ns 1.11 0.09 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark.html>) 98.86 ns 116.75 ns 1.18 0.11 False
[CreateRotationWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark.html>) 187.23 ns 294.83 ns 1.57 0.17 False
[CreateSkewFromScalarXYBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateSkewFromScalarXYBenchmark.html>) 67.42 ns 95.31 ns 1.41 0.15 False
[CreateTranslationFromVectorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromVectorBenchmark.html>) 73.00 ns 95.17 ns 1.30 0.16 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark.html>) 117.14 ns 144.29 ns 1.23 0.12 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark.html>) 117.77 ns 141.16 ns 1.20 0.06 False
[SubtractOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark.html>) 100.56 ns 123.71 ns 1.23 0.13 False
[LerpBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark.html>) 147.65 ns 169.96 ns 1.15 0.09 False
[CreateTranslationFromScalarXY - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromScalarXY.html>) 48.83 ns 72.61 ns 1.49 0.14 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark.html>) 100.41 ns 128.26 ns 1.28 0.11 False
[CreateSkewFromScalarXYWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateSkewFromScalarXYWithCenterBenchmark.html>) 86.53 ns 117.24 ns 1.35 0.13 False
[CreateRotationBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateRotationBenchmark.html>) 182.65 ns 274.71 ns 1.50 0.13 False
[CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark.html>) 67.08 ns 94.91 ns 1.41 0.16 False
[NegationOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark.html>) 90.71 ns 105.83 ns 1.17 0.09 False
[CreateScaleFromVectorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorBenchmark.html>) 66.58 ns 91.39 ns 1.37 0.12 False
[CreateScaleFromScalarXYBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYBenchmark.html>) 47.58 ns 72.53 ns 1.52 0.16 False
[CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark.html>) 93.20 ns 117.67 ns 1.26 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Matrix3x2.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 118.80788231893885 > 107.50606044597671. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.488233780595802 (T) = (0 -117.69150635181481) / Math.Sqrt((2.0713210577179835 / (44)) + (2.9367497776502627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15187466205889752 = (102.17388247906179 - 117.69150635181481) / 102.17388247906179 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.71125275200119 > 72.66646671775602. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -74.94521330056642 (T) = (0 -97.83250615475359) / Math.Sqrt((2.208753092726739 / (44)) + (1.3933984413564642 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4080147326514481 = (69.48258699717162 - 97.83250615475359) / 69.48258699717162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 70.42734771392635 > 48.4553299894017. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.31931732757535 (T) = (0 -71.30108315033883) / Math.Sqrt((1.5829880455541638 / (44)) + (0.6276514625629783 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5189118581744261 = (46.94221245730171 - 71.30108315033883) / 46.94221245730171 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 122.29718899620892 > 116.37322299337796. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.243836026085507 (T) = (0 -123.03106642088397) / Math.Sqrt((17.760174530492016 / (44)) + (0.639373245277059 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0902397785867204 = (112.84771372070954 - 123.03106642088397) / 112.84771372070954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 116.7528847814311 > 103.94751910932949. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.4444691158583 (T) = (0 -115.24432351162142) / Math.Sqrt((9.053256323964618 / (44)) + (1.2597204515218692 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15321069075511323 = (99.93345052685936 - 115.24432351162142) / 99.93345052685936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateRotationWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.83486142928194 > 196.6197960959834. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.532036030766726 (T) = (0 -293.80311980661554) / Math.Sqrt((15.586466213410324 / (44)) + (36.86727911101728 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5549912482445201 = (188.94197645054234 - 293.80311980661554) / 188.94197645054234 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateSkewFromScalarXYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 95.30889136783014 > 71.33475947549405. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -68.15209897749737 (T) = (0 -95.39480976326601) / Math.Sqrt((1.937758168498299 / (44)) + (1.780909067794046 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40494664738354963 = (67.8992401177091 - 95.39480976326601) / 67.8992401177091 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromVectorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 95.16773123456493 > 71.96156921736124. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.08573589241593 (T) = (0 -95.32002591908399) / Math.Sqrt((6.298417551616347 / (44)) + (11.231912466345687 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36679994865778315 = (69.73955918910416 - 95.32002591908399) / 69.73955918910416 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 144.28728491910962 > 123.39918205460098. IsChangePoint: Marked as a change because one of 10/7/2022 9:25:17 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -60.227545225631694 (T) = (0 -142.13094958830374) / Math.Sqrt((1.1538020063884156 / (44)) + (1.870543530105479 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19706299210664222 = (118.7330579305402 - 142.13094958830374) / 118.7330579305402 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.16418912319062 > 123.54327093971304. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.94641047093347 (T) = (0 -142.06378960301026) / Math.Sqrt((2.6528047953581146 / (44)) + (5.718932277660856 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19539966322203878 = (118.84208601840865 - 142.06378960301026) / 118.84208601840865 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.SubtractOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.70771932103143 > 106.13002326379494. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.9090370103389 (T) = (0 -122.97796439039617) / Math.Sqrt((6.9066402762276065 / (44)) + (3.118667931388949 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21415354896353173 = (101.28699495659089 - 122.97796439039617) / 101.28699495659089 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.96199389049727 > 154.45676234680656. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.82175032115205 (T) = (0 -171.8517649965186) / Math.Sqrt((21.286767282754354 / (44)) + (19.459553469819095 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16063295267156177 = (148.06728053080667 - 171.8517649965186) / 148.06728053080667 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateTranslationFromScalarXY ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 72.60873669164313 > 51.240081028980185. IsChangePoint: Marked as a change because one of 10/19/2022 1:55:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -69.58845100222547 (T) = (0 -72.96456779027906) / Math.Sqrt((1.4365257395944226 / (43)) + (1.2507126628813292 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.5018122816090391 = (48.584346182137324 - 72.96456779027906) / 48.584346182137324 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 128.2637598397293 > 105.89898060333785. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.683543817662486 (T) = (0 -124.69439278089968) / Math.Sqrt((3.1341999613163494 / (44)) + (24.664715752759662 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22812066341325232 = (101.53268851803453 - 124.69439278089968) / 101.53268851803453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateSkewFromScalarXYWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 117.23595926739254 > 92.33683318707544. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.586624234661066 (T) = (0 -117.15304056481195) / Math.Sqrt((2.9602428289765492 / (44)) + (3.5837953622206733 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3304658530229007 = (88.05415058089089 - 117.15304056481195) / 88.05415058089089 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateRotationBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 274.7063280438869 > 174.9901378753205. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.67406219678169 (T) = (0 -278.19052349034797) / Math.Sqrt((19.083023447248586 / (44)) + (94.59491562949485 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6618117767861755 = (167.4019448991681 - 278.19052349034797) / 167.4019448991681 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 94.90939240725814 > 70.34971273394585. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -68.75511080428451 (T) = (0 -95.86194329620803) / Math.Sqrt((1.1088097323693773 / (44)) + (2.17374867934527 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4200980846193952 = (67.50374803998153 - 95.86194329620803) / 67.50374803998153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.83134686788307 > 94.70151461813202. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.578374675553 (T) = (0 -105.88386268363402) / Math.Sqrt((1.6493271427652134 / (44)) + (3.0931216681544282 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17264216297448998 = (90.29511817573751 - 105.88386268363402) / 90.29511817573751 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.39103920144437 > 71.73911933631639. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/13/2022 1:28:42 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.03028850720812 (T) = (0 -92.9538939157408) / Math.Sqrt((5.983475803098272 / (44)) + (0.8742777701928445 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34444433992103746 = (69.13926531253813 - 92.9538939157408) / 69.13926531253813 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromScalarXYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 72.52544287028257 > 50.008968131695916. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.05234619929382 (T) = (0 -74.87299082585653) / Math.Sqrt((21.118160077748573 / (44)) + (27.310503660999064 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5277668621875048 = (49.00812596409574 - 74.87299082585653) / 49.00812596409574 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix3x2.CreateScaleFromVectorWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 117.66624444022251 > 97.99896139211421. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.54868260010437 (T) = (0 -120.75925025241779) / Math.Sqrt((10.512409704378216 / (44)) + (2.8662157844224905 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3004689953827414 = (92.85823090067372 - 120.75925025241779) / 92.85823090067372 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.Parse(value%3a%20%2212345%22).html>) 393.15 ns 441.45 ns 1.12 0.12 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParse(value%3a%20%2212345%22).html>) 350.33 ns 375.73 ns 1.07 0.16 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ParseSpan(value%3a%20%2212345%22).html>) 437.02 ns 468.38 ns 1.07 0.08 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryFormat(value%3a%2018446744073709551615).html>) 730.83 ns 1.03 μs 1.41 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryFormat(value%3a%2012345).html>) 342.44 ns 385.67 ns 1.13 0.09 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ToString(value%3a%2018446744073709551615).html>) 766.91 ns 1.02 μs 1.33 0.05 False
[TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%22FFFFFFFFFFFFFFFF%22).html>) 954.30 ns 1.36 μs 1.43 0.07 False
[TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%220%22).html>) 261.36 ns 299.36 ns 1.15 0.23 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParse(value%3a%20%2218446744073709551615%22).html>) 769.24 ns 860.80 ns 1.12 0.06 False
[TryParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.TryParseHex(value%3a%20%223039%22).html>) 435.65 ns 582.09 ns 1.34 0.08 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.Parse(value%3a%20%2218446744073709551615%22).html>) 822.06 ns 896.38 ns 1.09 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ToString(value%3a%2012345).html>) 364.12 ns 405.19 ns 1.11 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ToString(value%3a%200).html>) 67.31 ns 90.39 ns 1.34 0.14 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt64.ParseSpan(value%3a%20%2218446744073709551615%22).html>) 861.33 ns 1.05 μs 1.22 0.04 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt64.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 441.4523213257669 > 417.40566936202816. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.964742737947585 (T) = (0 -427.318367185819) / Math.Sqrt((35.485877389609556 / (43)) + (58.79219076742243 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.08518781159485288 = (393.77365154683037 - 427.318367185819) / 393.77365154683037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 375.73432120230507 > 363.6826061235694. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -3.7345034136525377 (T) = (0 -389.1740214844056) / Math.Sqrt((52.824588782349686 / (43)) + (1791.4569697551435 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.11370277344758299 = (349.44154828642195 - 389.1740214844056) / 349.44154828642195 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.ParseSpan(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.3803864873122 > 467.4966200457263. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.643716616467968 (T) = (0 -476.49884409113577) / Math.Sqrt((222.0709302170766 / (44)) + (80.68350028024281 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06991838768422973 = (445.35999154336173 - 476.49884409113577) / 445.35999154336173 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.027924642541023 > 773.9559141906705. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.04480873445678 (T) = (0 -1007.2321240170461) / Math.Sqrt((131.35421495612516 / (44)) + (192.10805598499047 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.352404600253039 = (744.7712939075998 - 1007.2321240170461) / 744.7712939075998 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 385.6715449831569 > 360.61669601840424. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.423492221310156 (T) = (0 -388.0974399042779) / Math.Sqrt((55.905880015962644 / (44)) + (32.16534403371642 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11953403977531075 = (346.6597942677729 - 388.0974399042779) / 346.6597942677729 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.ToString(value: 18446744073709551615) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0235360439099426 > 792.3073152278049. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.787226675974498 (T) = (0 -1020.4893697621761) / Math.Sqrt((2939.5695001472254 / (44)) + (23.872255017784234 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34701070786690297 = (757.5955883663323 - 1020.4893697621761) / 757.5955883663323 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3628096490066453 > 1003.1977702617347. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.5831641604807 (T) = (0 -1368.7281810982643) / Math.Sqrt((309.30003156537384 / (44)) + (1033.374771824308 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4088096015300409 = (971.549441181941 - 1368.7281810982643) / 971.549441181941 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryParseHex(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 299.3585894237046 > 277.20742022367943. IsChangePoint: Marked as a change because one of 10/22/2022 10:22:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.323255138300564 (T) = (0 -292.8671131100594) / Math.Sqrt((43.41447001969476 / (44)) + (11.963773002467944 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09119375023146879 = (268.3914868903301 - 292.8671131100594) / 268.3914868903301 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryParse(value: "18446744073709551615") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 860.8003114723145 > 808.8245067881871. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.78541542326186 (T) = (0 -858.2892385479929) / Math.Sqrt((145.3996195848536 / (44)) + (314.38517912491574 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10660245772469462 = (775.6075658034747 - 858.2892385479929) / 775.6075658034747 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.TryParseHex(value: "3039") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 582.0870498709727 > 463.4369748828188. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.114996779969324 (T) = (0 -578.5464539504586) / Math.Sqrt((77.3965205303618 / (44)) + (63.60745734490365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3043593611323154 = (443.54835882668254 - 578.5464539504586) / 443.54835882668254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.Parse(value: "18446744073709551615") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 896.3810328608507 > 867.768773968042. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.48267806600861 (T) = (0 -904.9061953345273) / Math.Sqrt((264.8161327676056 / (43)) + (161.85745825319 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.09725817367374982 = (824.6976117797278 - 904.9061953345273) / 824.6976117797278 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 405.1937310777431 > 382.0006751102977. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.169417833339 (T) = (0 -397.4933653252255) / Math.Sqrt((775.8377889361714 / (44)) + (8.749511390839384 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1225768743201216 = (354.09010680534794 - 397.4933653252255) / 354.09010680534794 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.ToString(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 90.386690320601 > 70.81544221340049. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/1/2022 11:48:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.7178721537782 (T) = (0 -93.61681677607321) / Math.Sqrt((28.55402867110342 / (44)) + (14.971129131649887 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42394210454527204 = (65.74481959431154 - 93.61681677607321) / 65.74481959431154 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt64.ParseSpan(value: "18446744073709551615") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0513560252392828 > 907.2439543929161. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.259428491306496 (T) = (0 -958.3904900101185) / Math.Sqrt((209.50999311814573 / (44)) + (776.882309158679 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09691169799794108 = (873.7170838448998 - 958.3904900101185) / 873.7170838448998 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BenchAssignRectangular - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchAssignRectangular.html>) 16.82 secs 20.51 secs 1.22 0.05 False
[BenchAssignJagged - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchAssignJagged.html>) 11.49 secs 15.87 secs 1.38 0.03 False
[BenchIDEAEncryption - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchIDEAEncryption.html>) 18.01 secs 22.26 secs 1.24 0.05 False
[BenchEmFloatClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/ByteMark.BenchEmFloatClass.html>) 8.48 secs 9.94 secs 1.17 0.01 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### ByteMark.BenchAssignRectangular ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.50889413333333 > 17.732861719789998. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.24354790471811 (T) = (0 -20792697575.632114) / Math.Sqrt((31991643233014124 / (44)) + (3.695218732659026E+17 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22802548801460085 = (16931812717.705494 - 20792697575.632114) / 16931812717.705494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchAssignJagged ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.869051267066668 > 12.065241700978847. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.61809249993824 (T) = (0 -15916473427.336737) / Math.Sqrt((39074059165618760 / (43)) + (21219081852210536 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.37183032072349265 = (11602363052.409071 - 15916473427.336737) / 11602363052.409071 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchIDEAEncryption ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.257401428714285 > 19.251883410778845. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.17479221367067 (T) = (0 -22406751110.11893) / Math.Sqrt((55580638792966290 / (44)) + (80549125587962990 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2230551475231795 = (18320311357.582733 - 22406751110.11893) / 18320311357.582733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### ByteMark.BenchEmFloatClass ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.944436214285716 > 8.879879460533653. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -65.02735660016077 (T) = (0 -9882817492.827204) / Math.Sqrt((3477733818220667.5 / (43)) + (5895993241052215 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1671944861097513 = (8467155740.057122 - 9882817492.827204) / 8467155740.057122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ArrayResize - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Array.ArrayResize.html>) 320.21 ns 364.87 ns 1.14 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Array.ArrayResize ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.8690768076923 > 358.04581778000005. IsChangePoint: Marked as a change because one of 12/3/2022 7:24:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.918429821926159 (T) = (0 -367.83793684761287) / Math.Sqrt((826.6355124303155 / (44)) + (16.024407559822986 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13655831769453697 = (323.6419382278222 - 367.83793684761287) / 323.6419382278222 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(MyEventsListerViewModel).SystemTextJsonReflection.html>) 10.87 ms 12.11 ms 1.11 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromString(MyEventsListerViewModel).JsonNet.html>) 20.12 ms 21.99 ms 1.09 0.00 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromString(MyEventsListerViewModel).SystemTextJsonSourceGen.html>) 11.05 ms 12.30 ms 1.11 0.01 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.113789307142858 > 11.355619553409092. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.95345511108546 (T) = (0 -12163539.680862112) / Math.Sqrt((12239958859.908546 / (44)) + (5272277853.605894 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.12270593835497202 = (10834127.856030187 - 12163539.680862112) / 10834127.856030187 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.992540000000005 > 21.185526136625874. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.50711574506528 (T) = (0 -21866208.781190477) / Math.Sqrt((39600795798.209724 / (44)) + (13124045659.167057 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09379637354425095 = (19991114.717574853 - 21866208.781190477) / 19991114.717574853 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.297556659999998 > 11.559411334350647. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.05559705775971 (T) = (0 -12378826.313573355) / Math.Sqrt((20904284228.707603 / (44)) + (8734713615.878775 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12088611777344278 = (11043785.909457935 - 12378826.313573355) / 11043785.909457935 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsTrue(Int32).HashSet(Size%3a%20512).html>) 118.10 μs 151.26 μs 1.28 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsTrue<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.2588501382743 > 124.13686811428433. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.55066240547958 (T) = (0 -141706.24294817558) / Math.Sqrt((3102381.5482252645 / (44)) + (21760832.605759848 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21783206424296908 = (116359.42845392502 - 141706.24294817558) / 116359.42845392502 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.ReadOnlyMemory<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlyMemory(Byte).ToArray(Size%3a%20512).html>) 473.24 ns 635.68 ns 1.34 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlyMemory<Byte>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 635.6803715341606 > 519.2195241943367. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.227943480512458 (T) = (0 -635.2485327601077) / Math.Sqrt((8072.416551546108 / (44)) + (166.44094526753014 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2896240614460182 = (492.58427455813893 - 635.2485327601077) / 492.58427455813893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchF.MatInv4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchF.MatInv4.Test.html>) 44.79 ms 47.43 ms 1.06 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.MatInv4*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.MatInv4.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.429645854166665 > 46.3734854225. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.00552109769487 (T) = (0 -47635910.12709706) / Math.Sqrt((2763742891655.1904 / (44)) + (99736873201.35936 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06483832407828198 = (44735345.310125306 - 47635910.12709706) / 44735345.310125306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(Location).SystemTextJsonSourceGen.html>) 36.51 μs 40.46 μs 1.11 0.02 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(Location).DataContractJsonSerializer.html>) 70.79 μs 78.95 μs 1.12 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(Location).JsonNet.html>) 35.37 μs 38.80 μs 1.10 0.01 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.458676293023686 > 38.80256255999614. IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.43104305451657 (T) = (0 -40291.97592161596) / Math.Sqrt((26737107.471366882 / (43)) + (147918.93952737277 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2911214229519087 = (31206.961022686664 - 40291.97592161596) / 31206.961022686664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<Location>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.94668351927349 > 73.92806670133258. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.25442356930389 (T) = (0 -79980.7015650096) / Math.Sqrt((788513.2987924267 / (44)) + (616478.406788673 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13537547536338104 = (70444.27442772752 - 79980.7015650096) / 70444.27442772752 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<Location>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.80414208346278 > 37.282944635479495. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.30754795393556 (T) = (0 -39185.32490316372) / Math.Sqrt((558387.8001197332 / (44)) + (71330.94691615365 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.086159294024901 = (36076.959538740884 - 39185.32490316372) / 36076.959538740884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Compare - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.Compare.html>) 61.73 ns 66.36 ns 1.08 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.Parse(text%3a%20%22Red%2c%20Orange%2c%20Yellow%2c%20Green%2c%20Blue%22).html>) 6.08 μs 7.53 μs 1.24 0.03 False
[TryParseGeneric - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Enum.TryParseGeneric(text%3a%20%22Red%2c%20Orange%2c%20Yellow%2c%20Green%2c%20Blue%22).html>) 6.20 μs 7.63 μs 1.23 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.Compare ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.36496462506894 > 64.49322662778877. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.331562473237998 (T) = (0 -66.88229704781622) / Math.Sqrt((1.8278216894216337 / (44)) + (1.209495516451921 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09331513497744559 = (61.17385089450533 - 66.88229704781622) / 61.17385089450533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.Parse(text: "Red, Orange, Yellow, Green, Blue") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.534093373074403 > 6.447357048916445. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.02731387329771 (T) = (0 -7516.46924166792) / Math.Sqrt((28829.52855412154 / (44)) + (1876.1062243424437 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21726371134525288 = (6174.889772538386 - 7516.46924166792) / 6174.889772538386 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Enum.TryParseGeneric(text: "Red, Orange, Yellow, Green, Blue") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.625484420289854 > 6.311868257317706. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.92734952109335 (T) = (0 -7231.180496491951) / Math.Sqrt((12601.463966170246 / (23)) + (179850.31554174743 / (36))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (36) - 2, .025) and -0.21108389287586923 = (5970.833679672359 - 7231.180496491951) / 5970.833679672359 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(Location).XmlSerializer.html>) 80.49 μs 87.39 μs 1.09 0.02 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(Location).DataContractSerializer.html>) 55.30 μs 61.36 μs 1.11 0.01 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<Location>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.39325877808989 > 85.82588287323536. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.10437704936154 (T) = (0 -88435.50056933511) / Math.Sqrt((628240.626697579 / (44)) + (366571.012103897 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08998401199417996 = (81134.67683579869 - 88435.50056933511) / 81134.67683579869 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_ToStream<Location>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.35628065096499 > 58.49782233468093. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.43417834006413 (T) = (0 -61937.37436578808) / Math.Sqrt((534651.712905305 / (44)) + (168460.5525123865 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10785564564747901 = (55907.44119879371 - 61937.37436578808) / 55907.44119879371 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.Logging.EventSourceLogger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NestedScopes_TwoMessages - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers%3a%20False%2c%20Json%3a%20False).html>) 1.71 μs 1.85 μs 1.08 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.EventSourceLogger*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.EventSourceLogger.NestedScopes_TwoMessages(HasSubscribers: False, Json: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8474157341317416 > 1.8156096703945523. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.90887023731317 (T) = (0 -1843.2167311106905) / Math.Sqrt((463.4243344572586 / (44)) + (377.5706784690889 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06899248357175955 = (1724.2560255915575 - 1843.2167311106905) / 1724.2560255915575 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Threading.Tests.Perf_Interlocked

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CompareExchange_object_NoMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_Interlocked.CompareExchange_object_NoMatch.html>) 83.99 ns 107.25 ns 1.28 0.10 False
[CompareExchange_object_Match - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_Interlocked.CompareExchange_object_Match.html>) 84.17 ns 101.86 ns 1.21 0.12 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Interlocked*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tests.Perf_Interlocked.CompareExchange_object_NoMatch ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 107.25173494003398 > 88.46781998446252. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.274748132570544 (T) = (0 -106.6474239616629) / Math.Sqrt((2.0896951561316843 / (44)) + (4.635498236321269 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2637337504089838 = (84.39073810219 - 106.6474239616629) / 84.39073810219 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tests.Perf_Interlocked.CompareExchange_object_Match ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 101.8570909662151 > 87.8899673480551. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.199358564839038 (T) = (0 -107.0585901998819) / Math.Sqrt((12.208800711167966 / (44)) + (19.3504665361186 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2609798609897208 = (84.90111024917837 - 107.0585901998819) / 84.90111024917837 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchF.Secant

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchF.Secant.Test.html>) 1.51 secs 1.99 secs 1.32 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Secant*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Secant.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9946362664000001 > 1.601878749855. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.45033005701874 (T) = (0 -2032887623.741719) / Math.Sqrt((678759409161178 / (44)) + (9067619831806644 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3269307697311763 = (1532022370.8080587 - 2032887623.741719) / 1532022370.8080587 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Decode_NoDecodingRequired - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired.html>) 1.68 μs 1.94 μs 1.16 0.05 False
[HtmlDecode_Entities - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities.html>) 5.49 μs 7.40 μs 1.35 0.06 False
[Decode_DecodingRequired - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired.html>) 2.59 μs 2.89 μs 1.11 0.03 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9365900159486775 > 1.7522797149231097. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.849785738299559 (T) = (0 -1926.8099825095576) / Math.Sqrt((393.6993596139681 / (43)) + (4130.200730829191 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16110070945856123 = (1659.4684395706365 - 1926.8099825095576) / 1659.4684395706365 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Tests.Perf_WebUtility.HtmlDecode_Entities ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.399694566079646 > 5.795965543360543. IsChangePoint: Marked as a change because one of 9/23/2022 10:31:05 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.9995464164826 (T) = (0 -7321.568579659917) / Math.Sqrt((11834.839405602741 / (44)) + (8451.487886768507 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32330455632416055 = (5532.791786040223 - 7321.568579659917) / 5532.791786040223 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Tests.Perf_WebUtility.Decode_DecodingRequired ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.8870601625824697 > 2.742750147167836. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.228884660372152 (T) = (0 -2899.344355664136) / Math.Sqrt((6313.330317178021 / (43)) + (1180.6277379600767 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1170426659791905 = (2595.5538172058937 - 2899.344355664136) / 2595.5538172058937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.DependencyInjection.GetService.Scoped.html>) 1.86 μs 1.99 μs 1.07 0.05 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.DependencyInjection.GetService.Scoped ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.99171760210718 > 1.9476260902041242. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.657550674711658 (T) = (0 -1998.7349987601083) / Math.Sqrt((1288.3427406317799 / (44)) + (766.2275375313376 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08600708399596517 = (1840.4437947179488 - 1998.7349987601083) / 1840.4437947179488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromReader(Mode%3a%20SourceGen).html>) 59.73 μs 68.81 μs 1.15 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromReader(Mode%3a%20Reflection).html>) 57.96 μs 66.81 μs 1.15 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 40.13 μs 46.77 μs 1.17 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromString(Mode%3a%20Reflection).html>) 49.51 μs 55.96 μs 1.13 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 40.13 μs 47.54 μs 1.18 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Location).DeserializeFromString(Mode%3a%20SourceGen).html>) 50.64 μs 57.13 μs 1.13 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.80855459070797 > 62.867109122252096. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.52282047985898 (T) = (0 -68563.46629533898) / Math.Sqrt((422747.07047876756 / (44)) + (153985.88526823028 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1529581176916804 = (59467.4387935347 - 68563.46629533898) / 59467.4387935347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.81058673104435 > 61.64805014839453. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.758030868426154 (T) = (0 -67589.78217580829) / Math.Sqrt((600899.2074979257 / (44)) + (187935.19695807336 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15087927092976589 = (58728.82054883501 - 67589.78217580829) / 58728.82054883501 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 46.77391142392392 > 41.437620032660675. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.52618192026467 (T) = (0 -46373.24859273293) / Math.Sqrt((280193.76841222186 / (44)) + (167186.25241318377 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17239491020419362 = (39554.290272939004 - 46373.24859273293) / 39554.290272939004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.9617450027571 > 51.28230096246816. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.92571202380914 (T) = (0 -55654.45627770946) / Math.Sqrt((660650.1780803355 / (44)) + (175054.64956175676 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1403280976082744 = (48805.651982476964 - 55654.45627770946) / 48805.651982476964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.538110758106924 > 42.66447971428976. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.638816472535325 (T) = (0 -47167.34959589445) / Math.Sqrt((362405.4738557937 / (44)) + (218405.52133657568 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16237329040338355 = (40578.48712226152 - 47167.34959589445) / 40578.48712226152 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Location>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.134504212287105 > 52.56869372009833. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.79141627376963 (T) = (0 -56840.73690591091) / Math.Sqrt((358898.07839139755 / (44)) + (290229.7132047912 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13887082437051243 = (49909.731366881286 - 56840.73690591091) / 49909.731366881286 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20).html>) 11.95 μs 13.15 μs 1.10 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20da).html>) 12.87 μs 14.63 μs 1.14 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20ja).html>) 11.96 μs 13.08 μs 1.09 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 4.60 μs 5.21 μs 1.13 0.01 False
[ToStringHebrewIsrael - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael.html>) 7.87 μs 9.69 μs 1.23 0.03 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 4.61 μs 5.30 μs 1.15 0.03 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 4.75 μs 5.28 μs 1.11 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring%3a%20fr).html>) 11.97 μs 13.10 μs 1.09 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20da).html>) 4.42 μs 4.97 μs 1.13 0.06 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.147007961044405 > 12.46300997612574. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.66474509678006 (T) = (0 -13142.722493412852) / Math.Sqrt((20957.72119828458 / (44)) + (9209.11091139232 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11068076170170917 = (11833.03334909346 - 13142.722493412852) / 11833.03334909346 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: da) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.63031102355632 > 13.62080460150036. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.55841895924315 (T) = (0 -14342.178867786666) / Math.Sqrt((23936.213141742515 / (44)) + (16515.65312684856 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10141147717455185 = (13021.63556945913 - 14342.178867786666) / 13021.63556945913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: ja) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.083846685526684 > 12.493776864915192. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.49320354573156 (T) = (0 -13089.82052424748) / Math.Sqrt((9028.67494422514 / (44)) + (24982.574660694307 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09992067062091989 = (11900.695089999628 - 13089.82052424748) / 11900.695089999628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.209607185746353 > 4.897257245840549. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.700094347918895 (T) = (0 -5357.385036284583) / Math.Sqrt((9287.90206435767 / (43)) + (4616.255723832223 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15093856795517305 = (4654.796689803207 - 5357.385036284583) / 4654.796689803207 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.685359662624379 > 8.454818098229342. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.58686961847455 (T) = (0 -9810.028038200482) / Math.Sqrt((18474.87714382799 / (44)) + (20082.95541218024 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22257895732549338 = (8024.044565318581 - 9810.028038200482) / 8024.044565318581 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.295225115300358 > 4.862456415500997. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.48033973017785 (T) = (0 -5348.255729653529) / Math.Sqrt((6157.08614062731 / (44)) + (7506.102179877486 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15573937403530408 = (4627.562104231085 - 5348.255729653529) / 4627.562104231085 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.276883181100586 > 4.878102746827558. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.25617093733421 (T) = (0 -5238.374262465963) / Math.Sqrt((12628.781824134618 / (43)) + (3843.4582587438194 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.13569252732679937 = (4612.4933786401525 - 5238.374262465963) / 4612.4933786401525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.Parse(culturestring: fr) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.098680518798966 > 12.50550958859524. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.293650440125674 (T) = (0 -13057.821863322339) / Math.Sqrt((141881.38431809712 / (44)) + (12596.14547757129 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09221881438700091 = (11955.31672895686 - 13057.821863322339) / 11955.31672895686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.972609444124425 > 4.655336194034992. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.92782074492272 (T) = (0 -5100.428288301044) / Math.Sqrt((7411.61628710596 / (43)) + (5546.205268995402 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16439420021477963 = (4380.327802526188 - 5100.428288301044) / 4380.327802526188 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchI.NDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchI.NDhrystone.Test.html>) 6.85 secs 7.83 secs 1.14 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.NDhrystone*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchI.NDhrystone.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.826784800066666 > 7.385867243321538. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.931004794582957 (T) = (0 -7956351286.166984) / Math.Sqrt((1.3462104953564326E+17 / (44)) + (18507957316869284 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14020385011855388 = (6978007735.493713 - 7956351286.166984) / 6978007735.493713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).GreaterThanOrEqualAllBenchmark.html>) 49.93 ns 665.48 ns 13.33 0.11 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).EqualsBenchmark.html>) 80.54 ns 88.79 ns 1.10 0.13 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).SquareRootBenchmark.html>) 374.08 ns 449.24 ns 1.20 0.08 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanBenchmark.html>) 380.15 ns 475.62 ns 1.25 0.07 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanOrEqualBenchmark.html>) 406.65 ns 504.92 ns 1.24 0.07 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).AndNotBenchmark.html>) 170.03 ns 403.38 ns 2.37 0.12 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).GreaterThanAnyBenchmark.html>) 285.39 ns 605.29 ns 2.12 0.11 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).NegateBenchmark.html>) 350.56 ns 485.45 ns 1.38 0.07 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).DivisionOperatorBenchmark.html>) 387.00 ns 500.65 ns 1.29 0.07 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).OneBenchmark.html>) 133.08 ns 230.62 ns 1.73 0.06 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).AddBenchmark.html>) 348.17 ns 486.84 ns 1.40 0.06 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanAnyBenchmark.html>) 47.26 ns 539.40 ns 11.41 0.10 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanOrEqualAllBenchmark.html>) 311.55 ns 720.69 ns 2.31 0.09 False
[GreaterThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).GreaterThanOrEqualBenchmark.html>) 417.34 ns 517.84 ns 1.24 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).EqualsAnyBenchmark.html>) 321.03 ns 606.17 ns 1.89 0.10 False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).EqualsAllBenchmark.html>) 47.62 ns 51.00 ns 1.07 0.11 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).ConditionalSelectBenchmark.html>) 351.43 ns 562.64 ns 1.60 0.08 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).MultiplyBenchmark.html>) 348.43 ns 486.61 ns 1.40 0.09 False
[DotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).DotBenchmark.html>) 334.04 ns 487.09 ns 1.46 0.10 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).DivideBenchmark.html>) 387.14 ns 521.71 ns 1.35 0.08 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).GreaterThanAllBenchmark.html>) 47.61 ns 642.39 ns 13.49 0.07 False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).EqualsStaticBenchmark.html>) 392.42 ns 492.14 ns 1.25 0.10 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanOrEqualAnyBenchmark.html>) 49.30 ns 562.35 ns 11.41 0.09 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).MultiplyOperatorBenchmark.html>) 352.93 ns 468.29 ns 1.33 0.10 False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).EqualityOperatorBenchmark.html>) 47.54 ns 51.20 ns 1.08 0.11 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).MaxBenchmark.html>) 430.35 ns 527.06 ns 1.22 0.07 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).OnesComplementBenchmark.html>) 65.74 ns 295.44 ns 4.49 0.08 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).AddOperatorBenchmark.html>) 354.89 ns 468.81 ns 1.32 0.07 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).MinBenchmark.html>) 466.95 ns 532.94 ns 1.14 0.06 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).SubtractionOperatorBenchmark.html>) 350.36 ns 468.00 ns 1.34 0.09 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).UnaryNegateOperatorBenchmark.html>) 348.75 ns 485.14 ns 1.39 0.10 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).LessThanAllBenchmark.html>) 283.89 ns 676.14 ns 2.38 0.08 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).GreaterThanOrEqualAnyBenchmark.html>) 310.38 ns 643.37 ns 2.07 0.08 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).SubtractBenchmark.html>) 346.87 ns 485.85 ns 1.40 0.09 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Byte).OnesComplementOperatorBenchmark.html>) 67.17 ns 298.65 ns 4.45 0.09 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 665.4767842558983 > 52.289647259833096. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -65.16774547138213 (T) = (0 -671.2754806849872) / Math.Sqrt((5.859417796756063 / (44)) + (1358.9146922303892 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -12.280393537801164 = (50.546354577089616 - 671.2754806849872) / 50.546354577089616 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.78779865291881 > 84.79512483750946. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.497700440583932 (T) = (0 -90.64684536197) / Math.Sqrt((3.030714146015513 / (44)) + (4.451314894079949 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11530010774315032 = (81.27574339197109 - 90.64684536197) / 81.27574339197109 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 449.2448154881716 > 389.09706913439715. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.17185877710246 (T) = (0 -469.1068563467953) / Math.Sqrt((45.63150584198616 / (43)) + (2711.3974288678114 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.25968281158652406 = (372.40077583973107 - 469.1068563467953) / 372.40077583973107 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 475.61610524491215 > 399.755035242154. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -104.41287818069092 (T) = (0 -478.44661601319626) / Math.Sqrt((27.675873910058545 / (44)) + (3.7642797391817324 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25741063751364945 = (380.5014859419802 - 478.44661601319626) / 380.5014859419802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 504.92157121545137 > 425.41516752406227. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.55896269976645 (T) = (0 -508.5276517767912) / Math.Sqrt((4.779599567446673 / (44)) + (90.21341759849352 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2534806053731858 = (405.6924770889395 - 508.5276517767912) / 405.6924770889395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 403.3833538474178 > 178.2463099782856. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -130.05848585971796 (T) = (0 -409.9423810384645) / Math.Sqrt((40.855952674839195 / (44)) + (36.08056422138927 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3769125692157718 = (172.46843083240503 - 409.9423810384645) / 172.46843083240503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 605.2907169506672 > 299.4599763348228. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.424593820637874 (T) = (0 -615.0067761792969) / Math.Sqrt((15.366285050147395 / (44)) + (715.0462417390662 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.147560586564474 = (286.37458706724743 - 615.0067761792969) / 286.37458706724743 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 485.4542316607995 > 365.3733145265698. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -91.4046343222075 (T) = (0 -484.57657677186484) / Math.Sqrt((87.56431273638938 / (43)) + (1.019185948766384 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.37730501824296836 = (351.82952966369095 - 484.57657677186484) / 351.82952966369095 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 500.65071291284227 > 407.036921677224. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.383097817515097 (T) = (0 -505.57782444878285) / Math.Sqrt((110.68517552267787 / (44)) + (173.33361117332169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29103635755667595 = (391.60618637077243 - 505.57782444878285) / 391.60618637077243 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 230.61906683892303 > 139.38833155761046. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -199.42474503462125 (T) = (0 -229.57703313865812) / Math.Sqrt((1.4213147234888588 / (44)) + (3.0408231492622626 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7274765916967345 = (132.89733374225733 - 229.57703313865812) / 132.89733374225733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 486.84182809212064 > 366.68944735233987. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.85156322203746 (T) = (0 -486.8834947362795) / Math.Sqrt((423.4526959056819 / (44)) + (15.378282379215216 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3770401206859095 = (353.5724830542779 - 486.8834947362795) / 353.5724830542779 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 539.3999589453377 > 49.963659560110074. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -305.3586933511838 (T) = (0 -539.1865563750487) / Math.Sqrt((1.411935354148678 / (44)) + (38.28145470375026 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -10.160950381136695 = (48.31009349224746 - 539.1865563750487) / 48.31009349224746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 720.6852741780658 > 327.6479905588962. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.50279893618449 (T) = (0 -714.6218126358407) / Math.Sqrt((22.580713872227367 / (44)) + (1166.1689051379951 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.2898055501539014 = (312.088427153917 - 714.6218126358407) / 312.088427153917 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 517.8388758080212 > 439.53136376999674. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.600579913698247 (T) = (0 -521.5882167447467) / Math.Sqrt((440.3632917588484 / (44)) + (42.39712578609941 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2339223133363535 = (422.7075003891007 - 521.5882167447467) / 422.7075003891007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 606.1718136708909 > 299.70091325956975. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.6169600655094 (T) = (0 -609.5075915209794) / Math.Sqrt((189.1059632015122 / (44)) + (88.81119115826147 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -1.0940386860087943 = (291.06797099469617 - 609.5075915209794) / 291.06797099469617 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.000313493631985 > 50.58542383961943. IsChangePoint: Marked as a change because one of 9/28/2022 12:34:15 AM, 9/28/2022 11:19:50 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.8705139622798 (T) = (0 -51.544123653119975) / Math.Sqrt((1.2178912314189902 / (44)) + (0.6160715747126931 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0642657023996164 = (48.431630876483794 - 51.544123653119975) / 48.431630876483794 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 562.6414658589608 > 373.3951771347983. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -141.8356630153684 (T) = (0 -566.4172932324004) / Math.Sqrt((40.79991030862648 / (44)) + (18.700686418218748 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.6043613212162907 = (353.0484596842505 - 566.4172932324004) / 353.0484596842505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 486.60624381525537 > 367.47217716242676. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -121.99293380016007 (T) = (0 -487.4339850506896) / Math.Sqrt((37.501206113269845 / (44)) + (5.841098750015147 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38674535437026863 = (351.49494715418535 - 487.4339850506896) / 351.49494715418535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.DotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 487.08828187350366 > 351.2925816154705. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.50371175684212 (T) = (0 -497.66930825295174) / Math.Sqrt((56.02994424800783 / (44)) + (229.85271280977938 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4779306418098665 = (336.73387246610463 - 497.66930825295174) / 336.73387246610463 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 521.7129104486327 > 407.7653990126332. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -108.18362009014405 (T) = (0 -518.6109427043299) / Math.Sqrt((48.959130933096475 / (44)) + (3.8678918405407146 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3231252761242289 = (391.95906242791574 - 518.6109427043299) / 391.95906242791574 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 642.3938669817169 > 51.18091401653285. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -704.1425732215582 (T) = (0 -637.9663662984989) / Math.Sqrt((0.733617990695301 / (44)) + (10.274965886555664 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -12.253488083536684 = (48.13573319547272 - 637.9663662984989) / 48.13573319547272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 492.13732133975947 > 412.8677612110662. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.468772217173886 (T) = (0 -496.3419007839247) / Math.Sqrt((62.780958786859 / (44)) + (75.22953227244287 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2528674684605795 = (396.16472873526584 - 496.3419007839247) / 396.16472873526584 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 562.3488304176966 > 51.54810312736792. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -498.13083650675117 (T) = (0 -565.6270910203427) / Math.Sqrt((2.1316909501164423 / (44)) + (15.343590604608877 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -10.305591447287586 = (50.03073865331007 - 565.6270910203427) / 50.03073865331007 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.2945918173983 > 367.175231327822. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -105.3535996182791 (T) = (0 -469.4566967983233) / Math.Sqrt((38.33148156455554 / (43)) + (5.0715172712664485 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.3356018854383569 = (351.4944849334675 - 469.4566967983233) / 351.4944849334675 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 51.19677312391973 > 50.25766267290412. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.15099714087197 (T) = (0 -51.935718008908694) / Math.Sqrt((1.5069859807407036 / (44)) + (1.349376759921815 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07397989892041211 = (48.35818441398727 - 51.935718008908694) / 48.35818441398727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 527.0606689658161 > 449.6683901976759. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.52665791498459 (T) = (0 -526.9220608080298) / Math.Sqrt((154.3138872177294 / (44)) + (0.9704780388439075 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2213377583427238 = (431.4302552334326 - 526.9220608080298) / 431.4302552334326 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 295.44230690740545 > 71.57320307642404. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.73551549963835 (T) = (0 -302.89927478843276) / Math.Sqrt((1.9105340057412172 / (43)) + (878.9439722082607 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -3.485053643566962 = (67.53526242052638 - 302.89927478843276) / 67.53526242052638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.8149702719926 > 368.8441111089387. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.61965046977145 (T) = (0 -470.4851550968282) / Math.Sqrt((461.11476897200384 / (44)) + (11.40247474497482 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32748400889708823 = (354.4186987892387 - 470.4851550968282) / 354.4186987892387 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 532.9398968255392 > 461.43518397787636. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.24574563762213 (T) = (0 -537.329470590265) / Math.Sqrt((88.58793533285197 / (44)) + (184.54301110131541 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2162067677953843 = (441.8076636461094 - 537.329470590265) / 441.8076636461094 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 467.9996873360613 > 368.22851805278106. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.75080776723941 (T) = (0 -469.37441720456405) / Math.Sqrt((83.39188116352517 / (43)) + (1.3314204181276483 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.32985432822978306 = (352.95175361752985 - 469.37441720456405) / 352.95175361752985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 485.1436708371949 > 366.31084642460087. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.63726790978657 (T) = (0 -488.6326806808821) / Math.Sqrt((13.086886926293978 / (44)) + (113.22022808649142 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3976904129026202 = (349.6000803683885 - 488.6326806808821) / 349.6000803683885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 676.1413191586574 > 297.97278910881755. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -127.05107310397685 (T) = (0 -681.5799614255382) / Math.Sqrt((38.159912305868055 / (44)) + (132.07636983731916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3794380289764503 = (286.4457712810153 - 681.5799614255382) / 286.4457712810153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 643.367974240984 > 326.4564174454304. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.82025022187115 (T) = (0 -647.4769098695828) / Math.Sqrt((25.007390740902373 / (44)) + (1385.9810726206335 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0769639942163702 = (311.742000185166 - 647.4769098695828) / 311.742000185166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 485.8520489123594 > 368.0711621675646. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.88457828315973 (T) = (0 -490.36860798173063) / Math.Sqrt((9.174857387870784 / (44)) + (180.82652379598855 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39827660601466763 = (350.69499544826596 - 490.36860798173063) / 350.69499544826596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Byte>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 298.6534694738017 > 70.50672782362783. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -217.63711749233238 (T) = (0 -296.4875896093737) / Math.Sqrt((2.527881788052647 / (43)) + (15.67709694425562 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -3.3716387670260985 = (67.82069731966115 - 296.4875896093737) / 67.82069731966115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Sort<IntClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).Array(Size%3a%20512).html>) 222.55 μs 364.58 μs 1.64 0.04 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).List(Size%3a%20512).html>) 222.28 μs 363.09 μs 1.63 0.04 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).Array_Comparison(Size%3a%20512).html>) 336.72 μs 374.57 μs 1.11 0.03 False
[LinqQuery - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).LinqQuery(Size%3a%20512).html>) 663.02 μs 735.95 μs 1.11 0.02 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).Array_ComparerStruct(Size%3a%20512).html>) 354.22 μs 408.96 μs 1.15 0.04 False
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).Array_ComparerClass(Size%3a%20512).html>) 326.40 μs 386.14 μs 1.18 0.03 False
[LinqOrderByExtension - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntClass).LinqOrderByExtension(Size%3a%20512).html>) 685.55 μs 731.67 μs 1.07 0.04 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntClass&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntClass>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.5832460153847 > 235.34532751500004. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -85.28892523489382 (T) = (0 -366707.14425192674) / Math.Sqrt((7090694.940819788 / (44)) + (39265580.57031835 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6332116116908257 = (224531.31096238253 - 366707.14425192674) / 224531.31096238253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 363.09469984285715 > 234.17707221288464. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -80.43190504807576 (T) = (0 -367541.48045501835) / Math.Sqrt((7266947.069788328 / (44)) + (45093047.38456585 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6385731955518165 = (224305.80547318346 - 367541.48045501835) / 224305.80547318346 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 374.5727002 > 342.92605125750003. IsChangePoint: Marked as a change because one of 10/23/2022 1:14:54 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.538678621131993 (T) = (0 -379806.2813117321) / Math.Sqrt((14426001.36448974 / (43)) + (42149595.584882185 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.15935360044672298 = (327601.76115844626 - 379806.2813117321) / 327601.76115844626 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.LinqQuery(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 735.9536769230768 > 702.682907528. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.93129033593076 (T) = (0 -733773.7178281783) / Math.Sqrt((64032401.01187608 / (44)) + (65443122.55458198 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10509925664116643 = (663988.9706001674 - 733773.7178281783) / 663988.9706001674 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 408.9619998142857 > 373.2099403684616. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.919989681511957 (T) = (0 -407626.9322764909) / Math.Sqrt((24967715.687502187 / (43)) + (52588862.727076076 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.14101145582154537 = (357250.5168083465 - 407626.9322764909) / 357250.5168083465 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 386.13864004000004 > 341.98931997. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.58771287319196 (T) = (0 -380924.28439226124) / Math.Sqrt((11986128.33975085 / (43)) + (29097342.985622354 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16164895235093402 = (327916.8664692981 - 380924.28439226124) / 327916.8664692981 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntClass>.LinqOrderByExtension(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 731.6685000285713 > 705.1433388461537. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.257859657662582 (T) = (0 -734501.9237746251) / Math.Sqrt((140547938.84402162 / (44)) + (194488217.30727556 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10430516808407306 = (665125.859230522 - 734501.9237746251) / 665125.859230522 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Reflection.Activator<EmptyClass>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateInstanceNames - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Activator(EmptyClass).CreateInstanceNames.html>) 59.34 μs 65.03 μs 1.10 0.00 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Activator&lt;EmptyClass&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Activator<EmptyClass>.CreateInstanceNames ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.03470491319445 > 62.55348634572427. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.36996340777208 (T) = (0 -64733.660353568215) / Math.Sqrt((751824.6118205679 / (44)) + (140389.62454705065 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08851754781580218 = (59469.56067310215 - 64733.660353568215) / 59469.56067310215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XDocument.GetElement.html>) 961.95 ns 1.07 μs 1.11 0.13 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XDocument*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XDocument.GetElement ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0721338873167434 > 1005.6126961742233. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.826832387787494 (T) = (0 -1064.6314114329905) / Math.Sqrt((881.0979871951006 / (43)) + (358.52839649254935 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1255753820434671 = (945.8552740378582 - 1064.6314114329905) / 945.8552740378582 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DateTime_ToString_Unspecified - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified.html>) 1.61 μs 1.89 μs 1.17 0.08 False
[DateTime_ToString_RoundtripKind - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind.html>) 1.59 μs 1.87 μs 1.17 0.09 False
[DateTime_ToString_Local - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local.html>) 12.87 μs 14.45 μs 1.12 0.16 False
[DateTime_ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Tests.Perf_XmlConvert.DateTime_ToString.html>) 1.70 μs 2.01 μs 1.18 0.12 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8903628707145528 > 1.7623854829601313. IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.310377041441573 (T) = (0 -1934.1045653408598) / Math.Sqrt((4100.9915400418395 / (44)) + (922.8317860448691 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16752312374914213 = (1656.5878019872332 - 1934.1045653408598) / 1656.5878019872332 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8672832331288343 > 1.7070291298465143. IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.920203753698475 (T) = (0 -1896.0489910679016) / Math.Sqrt((9884.673820605154 / (44)) + (2201.5235034828956 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16481007053727348 = (1627.7752390939934 - 1896.0489910679016) / 1627.7752390939934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Local ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.445461589993801 > 13.591484065747327. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 5:51:37 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.260657043923788 (T) = (0 -14570.432824901844) / Math.Sqrt((4255484.712228257 / (45)) + (45687.541963383104 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.1846397396871222 = (12299.463150501833 - 14570.432824901844) / 12299.463150501833 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0094163578613022 > 1.8693578265056807. IsChangePoint: Marked as a change because one of 10/14/2022 3:33:41 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.119272812231095 (T) = (0 -2065.022632075016) / Math.Sqrt((4771.396057472377 / (44)) + (1051.02201995773 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1758923359328751 = (1756.1324017277177 - 2065.022632075016) / 1756.1324017277177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Next_int_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Random.Next_int_unseeded.html>) 274.87 ns 291.87 ns 1.06 0.07 False
[NextBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Random.NextBytes.html>) 75.23 μs 82.39 μs 1.10 0.08 False
[NextBytes_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Random.NextBytes_unseeded.html>) 18.11 μs 21.06 μs 1.16 0.07 False
[NextBytes_span_unseeded - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Random.NextBytes_span_unseeded.html>) 18.07 μs 21.05 μs 1.16 0.06 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Random.Next_int_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 291.8736941274142 > 289.2088652436052. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.197438032047152 (T) = (0 -294.6806806131364) / Math.Sqrt((24.098291538266665 / (44)) + (23.287570975388544 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06381342336916186 = (277.0041006625624 - 294.6806806131364) / 277.0041006625624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.38856900087261 > 79.05836480163262. IsChangePoint: Marked as a change because one of 10/24/2022 11:57:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.447640713929143 (T) = (0 -82254.13374072756) / Math.Sqrt((1947211.2364884324 / (44)) + (1516633.5637928045 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08798968137242552 = (75601.9428759375 - 82254.13374072756) / 75601.9428759375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.06105612244898 > 19.039492605667384. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.862465219768284 (T) = (0 -21466.984932260904) / Math.Sqrt((134796.474716531 / (44)) + (1168653.153260375 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.17488761778409237 = (18271.52197990554 - 21466.984932260904) / 18271.52197990554 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Random.NextBytes_span_unseeded ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.052746317733988 > 18.985327037484673. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.0449623488401 (T) = (0 -21192.23192834149) / Math.Sqrt((44387.07997915526 / (44)) + (96968.16257262207 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.166802996282925 = (18162.647847025946 - 21192.23192834149) / 18162.647847025946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.28 ms 1.45 ms 1.13 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 1.12 ms 1.25 ms 1.12 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 1.14 ms 1.25 ms 1.10 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.20 ms 1.35 ms 1.12 0.04 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 1.20 ms 1.33 ms 1.10 0.02 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableSortedDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 1.30 ms 1.44 ms 1.11 0.01 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) graph_6.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.446515298076923 > 1.3591347658503603. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.134130490337895 (T) = (0 -1442602.1782479237) / Math.Sqrt((280567321.9930943 / (43)) + (626375496.4653506 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.11718659335351568 = (1291281.3193699287 - 1442602.1782479237) / 1291281.3193699287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2494138621794872 > 1.1877626947265625. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.78710303011159 (T) = (0 -1259659.4394175396) / Math.Sqrt((301544013.01933503 / (44)) + (280815628.82038337 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1105087357060432 = (1134308.4470350149 - 1259659.4394175396) / 1134308.4470350149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2548113905325444 > 1.1880812498046875. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.38619475833188 (T) = (0 -1259486.1037665638) / Math.Sqrt((232102979.18788454 / (44)) + (187346219.89486295 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11319711612359593 = (1131413.3728197026 - 1259486.1037665638) / 1131413.3728197026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3474281990327381 > 1.269177420430381. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.61503748398514 (T) = (0 -1340430.423261218) / Math.Sqrt((307032706.9701636 / (44)) + (95130652.52386212 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10405536208890759 = (1214097.1089756598 - 1340430.423261218) / 1214097.1089756598 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.328744391025641 > 1.279298377403846. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.795441940571003 (T) = (0 -1342881.4178551077) / Math.Sqrt((265164331.21631148 / (44)) + (328516796.07044095 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10812872118498848 = (1211846.0537861378 - 1342881.4178551077) / 1211846.0537861378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4371494768356643 > 1.3611560549479167. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.97901930630045 (T) = (0 -1436949.8004570429) / Math.Sqrt((177367540.47241136 / (44)) + (86312456.45030288 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11392064583345614 = (1289992.9683787217 - 1436949.8004570429) / 1289992.9683787217 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FirstTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstTenSegments.html>) 104.39 ns 119.12 ns 1.14 0.08 False
[SliceSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceSingleSegment.html>) 193.43 ns 241.27 ns 1.25 0.11 False
[IterateTryGetArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateTryGetArray.html>) 443.13 ns 470.40 ns 1.06 0.11 False
[IterateTryGetSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateTryGetSingleSegment.html>) 479.77 ns 562.71 ns 1.17 0.06 False
[SliceTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceTenSegments.html>) 509.99 ns 680.80 ns 1.33 0.04 False
[FirstSpanTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstSpanTenSegments.html>) 135.27 ns 159.04 ns 1.18 0.08 False
[IterateTryGetMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateTryGetMemory.html>) 693.29 ns 786.90 ns 1.14 0.08 False
[IterateGetPositionTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateGetPositionTenSegments.html>) 2.19 μs 2.72 μs 1.24 0.07 False
[IterateGetPositionSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateGetPositionSingleSegment.html>) 1.61 μs 1.74 μs 1.08 0.09 False
[IterateForEachSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateForEachSingleSegment.html>) 590.13 ns 676.45 ns 1.15 0.09 False
[FirstSpanArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstSpanArray.html>) 149.09 ns 170.70 ns 1.14 0.07 False
[SliceMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceMemory.html>) 214.74 ns 259.49 ns 1.21 0.11 False
[IterateForEachTenSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).IterateForEachTenSegments.html>) 3.59 μs 3.90 μs 1.09 0.10 False
[FirstArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstArray.html>) 140.69 ns 153.58 ns 1.09 0.11 False
[FirstSpanMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstSpanMemory.html>) 170.38 ns 180.97 ns 1.06 0.05 False
[FirstMemory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstMemory.html>) 151.43 ns 169.77 ns 1.12 0.06 False
[SliceArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).SliceArray.html>) 199.20 ns 238.40 ns 1.20 0.10 False
[FirstSpanSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstSpanSingleSegment.html>) 144.55 ns 155.95 ns 1.08 0.09 False
[FirstSingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Tests.ReadOnlySequenceTests(Byte).FirstSingleSegment.html>) 107.78 ns 123.71 ns 1.15 0.08 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.11919654688205 > 109.25402507044261. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.529481517540372 (T) = (0 -120.55321266561147) / Math.Sqrt((8.065931319056403 / (44)) + (25.445048423446842 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15090727877868235 = (104.74624228073343 - 120.55321266561147) / 104.74624228073343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 241.27074929533356 > 206.43403995152747. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.440829581065504 (T) = (0 -248.53800502204007) / Math.Sqrt((8.454618229952661 / (44)) + (131.10282281104227 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2653874969292677 = (196.4125658149543 - 248.53800502204007) / 196.4125658149543 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 470.4037372052345 > 460.4945237404345. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/19/2022 1:29:52 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.710128203719607 (T) = (0 -477.245814109192) / Math.Sqrt((263.86779540014174 / (43)) + (54.54771372015617 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.07450118113966166 = (444.1556905530851 - 477.245814109192) / 444.1556905530851 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 562.7091784211126 > 504.38378955688114. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.385797111691062 (T) = (0 -568.6720899067415) / Math.Sqrt((64.57613418306504 / (44)) + (79.76323719394541 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1733422665256717 = (484.6600230217646 - 568.6720899067415) / 484.6600230217646 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 680.7988995960276 > 540.577383594546. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -78.06373716036198 (T) = (0 -682.6338402588283) / Math.Sqrt((85.78486089674671 / (44)) + (33.676706248353014 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3134572081776894 = (519.7229388279234 - 682.6338402588283) / 519.7229388279234 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 159.0359583854419 > 142.55150195224013. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.937644323422646 (T) = (0 -151.83595886456658) / Math.Sqrt((16.611030132888907 / (43)) + (4.146948940664716 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.1048685455098681 = (137.4244560419613 - 151.83595886456658) / 137.4244560419613 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 786.9023442478539 > 733.2879955803606. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.069762506449813 (T) = (0 -766.0027088939929) / Math.Sqrt((474.4075857494595 / (44)) + (134.52411414811382 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0820484865397718 = (707.9190243531085 - 766.0027088939929) / 707.9190243531085 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7233486279455583 > 2.3316897665085095. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.775478666364755 (T) = (0 -2714.7565610329502) / Math.Sqrt((1526.7217534036956 / (44)) + (605.9423688826971 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21660333199893564 = (2231.4229212018345 - 2714.7565610329502) / 2231.4229212018345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.743904445678169 > 1.676340769492638. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.134556815510784 (T) = (0 -1731.1525669056743) / Math.Sqrt((1800.7904471576567 / (44)) + (176.31132316944522 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06775370502626939 = (1621.3032638112772 - 1731.1525669056743) / 1621.3032638112772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 676.4452490539478 > 618.5759129982184. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.681469917747766 (T) = (0 -679.1572513848101) / Math.Sqrt((327.2853466737163 / (44)) + (112.08705188188618 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14060609770640714 = (595.4354029410298 - 679.1572513848101) / 595.4354029410298 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 170.7025629472841 > 157.4817694967335. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.672907054855145 (T) = (0 -166.14968666700562) / Math.Sqrt((1.9611156638724652 / (44)) + (2.835797072275284 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10149112898416753 = (150.8406943052138 - 166.14968666700562) / 150.8406943052138 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 259.4915611814346 > 225.37990023624053. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.8473914662956 (T) = (0 -260.5219195417781) / Math.Sqrt((16.66642874039938 / (44)) + (8.412372515409377 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2056706056809496 = (216.08051014450848 - 260.5219195417781) / 216.08051014450848 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateForEachTenSegments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.9035039177380044 > 3.8733335725083515. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.619559115225236 (T) = (0 -3944.976506132639) / Math.Sqrt((4084.386639687546 / (44)) + (4015.2731547188005 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08694452227916571 = (3629.418452618532 - 3944.976506132639) / 3629.418452618532 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.581325355659 > 144.18975459951812. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.719723806229203 (T) = (0 -155.00620650375367) / Math.Sqrt((2.8416818071504304 / (44)) + (45.87655534977371 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12462333156701572 = (137.82944222557856 - 155.00620650375367) / 137.82944222557856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 180.9737639572294 > 172.93562228655108. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.473026056171292 (T) = (0 -181.6554441421511) / Math.Sqrt((15.039334781918075 / (44)) + (2.180079153211444 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09450589137873126 = (165.97027532974053 - 181.6554441421511) / 165.97027532974053 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstMemory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.7704283334817 > 159.2373230632877. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.262358782466448 (T) = (0 -170.22225148284616) / Math.Sqrt((10.757347843118128 / (44)) + (29.40046763799162 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10893732082393566 = (153.50033612032442 - 170.22225148284616) / 153.50033612032442 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.SliceArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 238.3958179519108 > 206.6515748681032. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.41778898899174 (T) = (0 -236.32663654070834) / Math.Sqrt((10.346697413026991 / (44)) + (8.912820354022214 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22357647999313596 = (193.1441478362138 - 236.32663654070834) / 193.1441478362138 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 155.9526925088075 > 149.81235279915305. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.959611829037275 (T) = (0 -156.24414158144407) / Math.Sqrt((17.004288302735844 / (44)) + (3.855791070282485 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10157484233116082 = (141.83706415335254 - 156.24414158144407) / 141.83706415335254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSingleSegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.70659219538551 > 113.6525740818569. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.232344704831426 (T) = (0 -123.98231868806471) / Math.Sqrt((3.4862893984383976 / (44)) + (1.5002025553125666 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14119633716888522 = (108.642408540886 - 123.98231868806471) / 108.642408540886 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ImmutableList(Size%3a%20512).html>) 1.53 ms 1.65 ms 1.08 0.04 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).Stack(Size%3a%20512).html>) 28.11 μs 29.86 μs 1.06 0.08 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).HashSet(Size%3a%20512).html>) 116.60 μs 134.03 μs 1.15 0.05 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ConcurrentDictionary(Size%3a%20512).html>) 416.65 μs 457.43 μs 1.10 0.03 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).ImmutableSortedSet(Size%3a%20512).html>) 2.85 ms 3.02 ms 1.06 0.03 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(Int32).Span(Size%3a%20512).html>) 8.28 μs 9.16 μs 1.11 0.07 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<Int32>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6512475004166667 > 1.602305752840909. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.69944468091073 (T) = (0 -1669820.1439757326) / Math.Sqrt((1846675212.3436556 / (44)) + (343566951.78058237 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08193149583090101 = (1543369.5667518629 - 1669820.1439757326) / 1543369.5667518629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.856306200491673 > 29.545060039990652. IsChangePoint: Marked as a change because one of 12/2/2022 12:08:23 PM, 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.610685210403428 (T) = (0 -30048.07031538789) / Math.Sqrt((1085778.716069031 / (44)) + (47086.37481053763 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.050192210844215844 = (28611.972175296567 - 30048.07031538789) / 28611.972175296567 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 134.03411237026862 > 122.46813565675231. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.054020703071185 (T) = (0 -135667.40190525402) / Math.Sqrt((11554032.86380166 / (44)) + (5674880.471409301 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1418649772775352 = (118812.12280344726 - 135667.40190525402) / 118812.12280344726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 457.43392829670324 > 443.04177365487203. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.552887911316624 (T) = (0 -456542.62451193674) / Math.Sqrt((44523818.81840783 / (44)) + (18302259.637959566 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.075882473289031 = (424342.46848195343 - 456542.62451193674) / 424342.46848195343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.016104859722222 > 3.0072902351562503. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.069246371852925 (T) = (0 -3060081.9152294975) / Math.Sqrt((1677710684.9624503 / (44)) + (5449988980.131587 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0557823426434152 = (2898402.247917707 - 3060081.9152294975) / 2898402.247917707 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.164677858401141 > 8.70037078958513. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.15990343479091 (T) = (0 -9361.58877010476) / Math.Sqrt((8613.382202464663 / (43)) + (259350.4280261837 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.12569131411190781 = (8316.301860684074 - 9361.58877010476) / 8316.301860684074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 1.08 ms 1.21 ms 1.12 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.26 ms 1.41 ms 1.12 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.10 ms 1.24 ms 1.12 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 939.02 μs 1.06 ms 1.13 0.02 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 1.23 ms 1.37 ms 1.12 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(IndexViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 931.26 μs 1.03 ms 1.10 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2087776438301283 > 1.1419874509795673. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.71355130639226 (T) = (0 -1205847.7788344135) / Math.Sqrt((116583038.00641456 / (44)) + (132474460.56359395 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11154842252360493 = (1084836.030891678 - 1205847.7788344135) / 1084836.030891678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4125808548951049 > 1.318970550721154. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.15706172642815 (T) = (0 -1406011.7749906343) / Math.Sqrt((194921458.52422404 / (43)) + (35793703.13392177 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.12138065144537635 = (1253822.0390893936 - 1406011.7749906343) / 1253822.0390893936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2359870562130175 > 1.1629317887019233. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.095323640594625 (T) = (0 -1228812.6709367074) / Math.Sqrt((196031674.77355674 / (44)) + (222815600.82095772 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10728941694528549 = (1109748.4109679942 - 1228812.6709367074) / 1109748.4109679942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0628233977564103 > 996.7388966381505. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.669784626692284 (T) = (0 -1059078.8743650794) / Math.Sqrt((132240138.14767024 / (44)) + (93717978.78353854 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1249465263163628 = (941448.1929492534 - 1059078.8743650794) / 941448.1929492534 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3712504348958332 > 1.2944729239460058. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.10726251843644 (T) = (0 -1380837.9804592684) / Math.Sqrt((142485331.7520179 / (44)) + (187740880.42679924 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11939925351718246 = (1233552.707955306 - 1380837.9804592684) / 1233552.707955306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.026361219551282 > 976.7971142279412. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.09206957358028 (T) = (0 -1034829.4131948897) / Math.Sqrt((118365802.03074323 / (44)) + (51677421.20920584 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12400633343530987 = (920661.5500396087 - 1034829.4131948897) / 920661.5500396087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 798.93 μs 913.06 μs 1.14 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 831.33 μs 938.52 μs 1.13 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 866.06 μs 976.57 μs 1.13 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 1.03 ms 1.18 ms 1.14 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 849.75 μs 974.66 μs 1.15 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 1.07 ms 1.20 ms 1.12 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 1.02 ms 1.14 ms 1.12 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToString(Mode%3a%20Reflection).html>) 1.07 ms 1.18 ms 1.11 0.01 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 913.0605189479638 > 838.4834441097861. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.764353710723945 (T) = (0 -919813.078129085) / Math.Sqrt((63305214.11012745 / (44)) + (121010716.63952899 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14934290614164805 = (800294.7364219646 - 919813.078129085) / 800294.7364219646 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 938.5175945378151 > 861.4110673424846. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.20172114369569 (T) = (0 -939700.6839098525) / Math.Sqrt((80886034.18229638 / (44)) + (92236412.59209165 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.145375638881524 = (820430.1296537824 - 939700.6839098525) / 820430.1296537824 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 976.56610546875 > 908.8168565604968. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.26782023624387 (T) = (0 -983569.25896544) / Math.Sqrt((90092682.93276301 / (44)) + (74853273.01874673 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13558819462855193 = (866131.9865932236 - 983569.25896544) / 866131.9865932236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1753115430484695 > 1.088409301201923. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.93457638528816 (T) = (0 -1171568.846570756) / Math.Sqrt((99198242.30293508 / (44)) + (57357203.64735166 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13341102428123183 = (1033666.3588689924 - 1171568.846570756) / 1033666.3588689924 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 974.6643626802885 > 896.1288664563301. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.25520659621987 (T) = (0 -971775.7474338072) / Math.Sqrt((96334812.28015296 / (44)) + (59913762.119732134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1375561091661518 = (854266.2112255153 - 971775.7474338072) / 854266.2112255153 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1995310680473374 > 1.1252031232812498. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.26811002297665 (T) = (0 -1205063.133089867) / Math.Sqrt((84315609.93648167 / (44)) + (105827499.74800707 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.13075189718290117 = (1065718.4269087685 - 1205063.133089867) / 1065718.4269087685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1376899048763738 > 1.067994171938101. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.89050461239 (T) = (0 -1141163.2330305362) / Math.Sqrt((125743136.13027813 / (44)) + (33819045.0977777 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13464234542372042 = (1005747.0864128384 - 1141163.2330305362) / 1005747.0864128384 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1834475120192307 > 1.1105866833653846. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.59945964381285 (T) = (0 -1192937.6242210248) / Math.Sqrt((80861284.66377968 / (44)) + (37315200.09536122 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12691642396323707 = (1058585.711286023 - 1192937.6242210248) / 1058585.711286023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(LoginViewModel).SystemTextJsonReflection.html>) 17.74 μs 19.41 μs 1.09 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToString(LoginViewModel).JsonNet.html>) 17.64 μs 18.82 μs 1.07 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.412737264016936 > 18.712057101705334. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.61639286788852 (T) = (0 -19526.59365243209) / Math.Sqrt((39439.13070282652 / (44)) + (12641.679697665575 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09498334648944139 = (17832.776831752828 - 19526.59365243209) / 17832.776831752828 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<LoginViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.82159018309506 > 18.468882536435817. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.864700736689677 (T) = (0 -18923.890455937046) / Math.Sqrt((76591.80251291273 / (44)) + (32828.27726069948 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08583525379334123 = (17427.957316569762 - 18923.890455937046) / 17427.957316569762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EnumerateUsingIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase%3a%20ArrayOfStrings).html>) 77.26 μs 102.03 μs 1.32 0.12 False
[EnumerateUsingIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase%3a%20ArrayOfNumbers).html>) 76.27 μs 102.14 μs 1.34 0.11 False
[EnumerateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase%3a%20ArrayOfNumbers).html>) 74.77 μs 96.93 μs 1.30 0.11 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase%3a%20Json400KB).html>) 23.63 ms 30.78 ms 1.30 0.04 False
[EnumerateUsingIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase%3a%20Json400KB).html>) 5.56 ms 8.78 ms 1.58 0.08 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase%3a%20ArrayOfNumbers).html>) 313.76 μs 392.07 μs 1.25 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase%3a%20ArrayOfStrings).html>) 233.78 μs 308.83 μs 1.32 0.07 False
[EnumerateArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase%3a%20ArrayOfStrings).html>) 78.69 μs 96.14 μs 1.22 0.15 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.0306412627551 > 81.23287890207908. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.121086939375324 (T) = (0 -103364.3911135146) / Math.Sqrt((3157721.7983731185 / (44)) + (4856568.814980627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.33368923886104984 = (77502.60563081862 - 103364.3911135146) / 77502.60563081862 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.14492839635854 > 80.46771840948375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.563760901733204 (T) = (0 -103162.0356627517) / Math.Sqrt((3213672.382583608 / (44)) + (1882371.9467795098 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3308902742106482 = (77513.55439420967 - 103162.0356627517) / 77513.55439420967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.92687968611358 > 79.83403954882968. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.251935694943384 (T) = (0 -97231.42722555791) / Math.Sqrt((3102825.754722123 / (44)) + (3224226.7929935358 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28400412785546797 = (75725.16716745526 - 97231.42722555791) / 75725.16716745526 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 30.784224975 > 24.654651491500005. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -63.60384238283114 (T) = (0 -31013187.282353476) / Math.Sqrt((477256852301.6172 / (44)) + (47622175983.10964 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3207393165772289 = (23481687.031719413 - 31013187.282353476) / 23481687.031719413 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.782069191964286 > 5.808336516041667. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.98161555951896 (T) = (0 -8797941.168488247) / Math.Sqrt((2134138812.3633072 / (44)) + (33595644913.779274 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5863515377000835 = (5546022.40386367 - 8797941.168488247) / 5546022.40386367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 392.0669659189896 > 328.58254814448543. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.16798189543076 (T) = (0 -397803.7956824376) / Math.Sqrt((39306557.91554182 / (44)) + (26407358.757818043 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26508482097328584 = (314448.3192647822 - 397803.7956824376) / 314448.3192647822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.Parse(TestCase: ArrayOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 308.8294924019608 > 246.62471921097637. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.03877130862924 (T) = (0 -302772.2063959052) / Math.Sqrt((51831886.80642382 / (44)) + (18816300.18256385 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2742987003288499 = (237599.08592684803 - 302772.2063959052) / 237599.08592684803 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.14352720450283 > 79.01177425804364. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.74199669426434 (T) = (0 -96735.21807794663) / Math.Sqrt((2359068.269361178 / (44)) + (1899399.9849413761 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2792735802176788 = (75617.30311157239 - 96735.21807794663) / 75617.30311157239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TwoArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage.html>) 3.40 μs 4.01 μs 1.18 0.08 False
[TwoArguments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments.html>) 4.46 μs 5.48 μs 1.23 0.10 False
[FourArguments_DefineMessage - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage.html>) 5.72 μs 6.71 μs 1.17 0.03 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.013491305998482 > 3.5866833900689774. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.873589929267744 (T) = (0 -4073.577116602299) / Math.Sqrt((8357.101917664835 / (43)) + (4616.066210344943 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.20456524557092606 = (3381.7820425920986 - 4073.577116602299) / 3381.7820425920986 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.479460193947892 > 4.909299299315893. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.53756519689174 (T) = (0 -5388.970928264402) / Math.Sqrt((13492.8761151506 / (44)) + (6592.344203965628 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1760836061989992 = (4582.13251154915 - 5388.970928264402) / 4582.13251154915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.706017164258394 > 6.096719440893485. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.606059577302226 (T) = (0 -6717.538065804679) / Math.Sqrt((18530.068626845426 / (43)) + (12143.319300681527 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.15921590311349415 = (5794.898127054932 - 6717.538065804679) / 5794.898127054932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int16.TryParse(value%3a%20%22-32768%22).html>) 419.44 ns 451.20 ns 1.08 0.10 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int16.Parse(value%3a%20%2232767%22).html>) 403.36 ns 432.35 ns 1.07 0.10 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int16.TryParse(value: "-32768") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 451.19635500321476 > 441.1314416247335. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.74625433464484 (T) = (0 -452.5407898777023) / Math.Sqrt((48.67130478872106 / (43)) + (48.56819546311244 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.08384066371933364 = (417.53442643935546 - 452.5407898777023) / 417.53442643935546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int16.Parse(value: "32767") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 432.3513424066638 > 423.03320404788576. IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.08840243718749 (T) = (0 -432.12051725954603) / Math.Sqrt((92.74296820511096 / (43)) + (64.10133768410668 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.06449806238423977 = (405.93828446403353 - 432.12051725954603) / 405.93828446403353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(MyEventsListerViewModel).XmlSerializer.html>) 13.94 ms 15.52 ms 1.11 0.02 False
[DataContractSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(MyEventsListerViewModel).DataContractSerializer.html>) 24.88 ms 27.67 ms 1.11 0.00 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.519976185714285 > 14.598873619166667. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.267321453373132 (T) = (0 -15603789.625059826) / Math.Sqrt((82084523856.20515 / (44)) + (21610520155.96002 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11627734248917244 = (13978416.502000423 - 15603789.625059826) / 13978416.502000423 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Xml_ToStream<MyEventsListerViewModel>.DataContractSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.667277746031743 > 26.167435015555554. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.157971249093166 (T) = (0 -27748213.071764007) / Math.Sqrt((51400745990.114655 / (44)) + (51068804640.22469 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1145028489271036 = (24897390.884622976 - 27748213.071764007) / 24897390.884622976 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Tests.Perf_Dictionary

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Clone - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_Dictionary.Clone(Items%3a%203000).html>) 173.56 μs 240.95 μs 1.39 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_Dictionary*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_Dictionary.Clone(Items: 3000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.95006875000004 > 181.34749201161858. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.7576522530044 (T) = (0 -242018.98199496057) / Math.Sqrt((962977555.1710757 / (44)) + (5019084.682426005 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26506889196689026 = (191308.93466100245 - 242018.98199496057) / 191308.93466100245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 294.46 μs 329.87 μs 1.12 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 242.89 μs 270.08 μs 1.11 0.08 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20Reflection).html>) 288.06 μs 322.46 μs 1.12 0.04 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 276.82 μs 306.66 μs 1.11 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToWriter(Mode%3a%20Reflection).html>) 275.52 μs 299.72 μs 1.09 0.05 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 269.97 μs 301.93 μs 1.12 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToWriter(Mode%3a%20SourceGen).html>) 245.06 μs 274.49 μs 1.12 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(HashSet(String)).SerializeToString(Mode%3a%20SourceGen).html>) 260.35 μs 285.86 μs 1.10 0.02 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) graph_6.png>) graph_7.png>) graph_8.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 329.8662396367522 > 312.66314714259795. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.31398950258766 (T) = (0 -333758.0917138819) / Math.Sqrt((11472201.331600195 / (44)) + (10502660.381579528 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11938901775917611 = (298160.94889157306 - 333758.0917138819) / 298160.94889157306 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 270.08205026169946 > 256.3584127640721. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.956574876203426 (T) = (0 -272475.91253887006) / Math.Sqrt((31980860.191463005 / (44)) + (3362147.7418415546 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11117985589669409 = (245213.14987211398 - 272475.91253887006) / 245213.14987211398 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.4620059840426 > 304.58270445312496. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.104519876840786 (T) = (0 -324316.7714892918) / Math.Sqrt((7485171.138122332 / (44)) + (32505264.370395985 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12222163551784716 = (288995.29399968876 - 324316.7714892918) / 288995.29399968876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.6559051216641 > 289.3590359286039. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.81088205123526 (T) = (0 -302964.8918961779) / Math.Sqrt((6607881.504722777 / (44)) + (5446932.750646575 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10392976480010499 = (274442.1806137617 - 302964.8918961779) / 274442.1806137617 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 299.7198115566038 > 278.5976430011917. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.42068434668301 (T) = (0 -300957.80030097783) / Math.Sqrt((12602088.899744367 / (44)) + (7730524.254447239 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12906412657193428 = (266555.09923492675 - 300957.80030097783) / 266555.09923492675 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 301.92840629006406 > 281.24637127174844. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.71858411541934 (T) = (0 -302344.16671237076) / Math.Sqrt((16626283.442717321 / (44)) + (6600021.335373917 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12338280584129449 = (269137.2568106444 - 302344.16671237076) / 269137.2568106444 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 274.4903744971264 > 255.65789790861632. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.02625807003165 (T) = (0 -272481.82240823936) / Math.Sqrt((7319133.782685283 / (44)) + (6013871.400175378 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11745469905703003 = (243841.49320610004 - 272481.82240823936) / 243841.49320610004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 285.8649256060606 > 272.78561766649597. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.77080166078315 (T) = (0 -289695.7620612918) / Math.Sqrt((14902400.026984911 / (44)) + (15651021.951218402 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10676411135188899 = (261750.2312280749 - 289695.7620612918) / 261750.2312280749 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.ReadOnlyMemory<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlyMemory(Char).ToArray(Size%3a%20512).html>) 827.17 ns 1.06 μs 1.29 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory&lt;Char&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlyMemory<Char>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.064113915828148 > 849.0690054382296. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.441386860861693 (T) = (0 -1071.8227742683298) / Math.Sqrt((26956.437174727642 / (44)) + (128.0510923588757 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3206514549777515 = (811.586410803891 - 1071.8227742683298) / 811.586410803891 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.Count(Int32).Dictionary(Size%3a%20512).html>) 8.71 μs 9.31 μs 1.07 0.06 False
[Bag - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Concurrent.Count(Int32).Bag(Size%3a%20512).html>) 685.33 ns 776.15 ns 1.13 0.05 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Concurrent.Count<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.312159255165621 > 9.076269225917637. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.407087116113726 (T) = (0 -9328.951369936123) / Math.Sqrt((23360.72660429789 / (44)) + (34497.014395854436 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0695833824903214 = (8722.042173295022 - 9328.951369936123) / 8722.042173295022 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Concurrent.Count<Int32>.Bag(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 776.1531852393437 > 731.6049904722767. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.65924457126823 (T) = (0 -753.8863545759626) / Math.Sqrt((396.23330852165793 / (44)) + (145.61022772191592 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07832866342274644 = (699.1248402718292 - 753.8863545759626) / 699.1248402718292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.TryAddGiventSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddGiventSize(String).ConcurrentDictionary(Count%3a%20512).html>) 489.63 μs 525.13 μs 1.07 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<String>.ConcurrentDictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 525.1311120833333 > 510.55761133729885. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.579640132200293 (T) = (0 -521668.62766137876) / Math.Sqrt((105028487.33531234 / (44)) + (76737354.21253085 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07677716413574548 = (484472.2241858519 - 521668.62766137876) / 484472.2241858519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).MultiplyBenchmark.html>) 137.30 ns 171.64 ns 1.25 0.07 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).AbsBenchmark.html>) 111.54 ns 145.11 ns 1.30 0.11 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).LessThanOrEqualAnyBenchmark.html>) 53.30 ns 224.91 ns 4.22 0.13 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).GreaterThanAnyBenchmark.html>) 108.92 ns 278.49 ns 2.56 0.08 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).DivideBenchmark.html>) 146.46 ns 177.87 ns 1.21 0.11 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).OnesComplementBenchmark.html>) 66.90 ns 158.63 ns 2.37 0.11 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).LessThanAllBenchmark.html>) 108.94 ns 354.88 ns 3.26 0.07 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).AddOperatorBenchmark.html>) 142.05 ns 155.20 ns 1.09 0.06 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).GreaterThanAllBenchmark.html>) 51.15 ns 308.86 ns 6.04 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).LessThanOrEqualAllBenchmark.html>) 115.49 ns 393.04 ns 3.40 0.12 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).AddBenchmark.html>) 136.67 ns 169.15 ns 1.24 0.09 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).EqualsBenchmark.html>) 80.78 ns 92.94 ns 1.15 0.09 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).SubtractBenchmark.html>) 136.92 ns 170.47 ns 1.25 0.08 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).AndNotBenchmark.html>) 171.00 ns 271.77 ns 1.59 0.08 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).MultiplyOperatorBenchmark.html>) 136.54 ns 151.55 ns 1.11 0.05 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).UnaryNegateOperatorBenchmark.html>) 135.70 ns 168.08 ns 1.24 0.12 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).LessThanAnyBenchmark.html>) 51.36 ns 221.25 ns 4.31 0.10 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).OneBenchmark.html>) 54.27 ns 91.91 ns 1.69 0.11 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).NegateBenchmark.html>) 134.64 ns 168.17 ns 1.25 0.09 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).OnesComplementOperatorBenchmark.html>) 67.98 ns 158.08 ns 2.33 0.08 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).GreaterThanOrEqualAnyBenchmark.html>) 115.39 ns 283.35 ns 2.46 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).EqualsAnyBenchmark.html>) 108.45 ns 260.69 ns 2.40 0.09 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).DivisionOperatorBenchmark.html>) 146.26 ns 161.61 ns 1.10 0.12 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).GreaterThanOrEqualAllBenchmark.html>) 55.13 ns 322.63 ns 5.85 0.14 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int32).SubtractionOperatorBenchmark.html>) 135.43 ns 153.77 ns 1.14 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int32>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.64187207946458 > 142.61412630151574. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.59618652606496 (T) = (0 -170.8303451897353) / Math.Sqrt((5.443960731537525 / (44)) + (8.17532075854896 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24862993233119018 = (136.81423195645752 - 170.8303451897353) / 136.81423195645752 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 145.1115466016354 > 117.29750087212575. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.84577497033762 (T) = (0 -146.8823683719331) / Math.Sqrt((24.647377120605793 / (44)) + (18.488330875000937 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.308216715209181 = (112.27678615041005 - 146.8823683719331) / 112.27678615041005 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 224.91496009795128 > 56.17195466639833. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.06653206520839 (T) = (0 -228.68670779944023) / Math.Sqrt((1.9918345947790075 / (44)) + (47.02407969511383 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.250419892500975 = (53.80332145605489 - 228.68670779944023) / 53.80332145605489 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 278.4939456175399 > 114.46605412123189. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -309.49318949403346 (T) = (0 -278.58437377388407) / Math.Sqrt((7.736506499853953 / (44)) + (1.823838329108586 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5372266153088305 = (109.79877480907429 - 278.58437377388407) / 109.79877480907429 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 177.86962269775643 > 153.6203520113117. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.146894426976978 (T) = (0 -181.0966213044456) / Math.Sqrt((20.32156965295053 / (43)) + (14.889314840962314 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.221674717828787 = (148.23636657251802 - 181.0966213044456) / 148.23636657251802 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.63030229459324 > 70.26671621957784. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.5054143516304 (T) = (0 -164.90117633850733) / Math.Sqrt((3.333037988155323 / (44)) + (118.09637884068697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.438728935285226 = (67.61767326930125 - 164.90117633850733) / 67.61767326930125 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 354.8846922268559 > 115.12472186514245. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -149.81713407898818 (T) = (0 -360.5203835697229) / Math.Sqrt((2.867006480390646 / (44)) + (41.16143044849401 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.294998955994411 = (109.41441511350041 - 360.5203835697229) / 109.41441511350041 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 155.19819586603083 > 145.74404524722067. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.314627154745466 (T) = (0 -153.76611895070695) / Math.Sqrt((5.33489207454611 / (44)) + (3.5269494646276858 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11674713636096101 = (137.6910796939853 - 153.76611895070695) / 137.6910796939853 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 308.85992659918674 > 54.103170016232326. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -260.47831589698654 (T) = (0 -311.38442092333173) / Math.Sqrt((1.1336095652454126 / (44)) + (14.518187611266814 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -5.018793932432205 = (51.73535170317764 - 311.38442092333173) / 51.73535170317764 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 393.04415783515753 > 120.89801791191411. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -115.46319052814292 (T) = (0 -369.8172835711994) / Math.Sqrt((13.023012299424613 / (44)) + (67.9900225484656 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.1854329863551487 = (116.09639416535128 - 369.8172835711994) / 116.09639416535128 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.15343601922208 > 145.78340899216673. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.978131584583277 (T) = (0 -172.71319423136703) / Math.Sqrt((6.412691786419167 / (44)) + (25.481575129137926 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2567216903864953 = (137.43153758908258 - 172.71319423136703) / 137.43153758908258 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.93762678701205 > 85.4318078122083. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.998640625542349 (T) = (0 -89.82659001908151) / Math.Sqrt((8.652363078159738 / (44)) + (14.346215468615485 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09130181677307286 = (82.31140884993157 - 89.82659001908151) / 82.31140884993157 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 170.46998886881028 > 143.61768308844898. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.61149525317957 (T) = (0 -171.45723695339487) / Math.Sqrt((8.919686147749342 / (44)) + (19.51896760766552 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2460979443078461 = (137.59531322285585 - 171.45723695339487) / 137.59531322285585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 271.7723443519088 > 179.90951642607237. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.00534372917349 (T) = (0 -275.22550755429717) / Math.Sqrt((95.03866396836187 / (44)) + (33.630369707550365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5963640040138734 = (172.40773837437723 - 275.22550755429717) / 172.40773837437723 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.5537920778181 > 143.3582327052978. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.60416834737867 (T) = (0 -153.11173927426316) / Math.Sqrt((0.774239300254432 / (44)) + (2.209802150129858 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12403370995468659 = (136.21632333467613 - 153.11173927426316) / 136.21632333467613 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 168.0773857325881 > 142.72773185413845. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.29108917701309 (T) = (0 -168.9495494026665) / Math.Sqrt((13.071590494099485 / (44)) + (8.095093447118922 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2362106891446265 = (136.6672775815974 - 168.9495494026665) / 136.6672775815974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 221.25275516421024 > 53.67271841334459. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -290.50328686462547 (T) = (0 -220.41152377977636) / Math.Sqrt((1.0911547194750897 / (44)) + (4.372675108162647 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -3.2602982251343153 = (51.736172477180844 - 220.41152377977636) / 51.736172477180844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.91446178250514 > 58.55662231941032. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.95985235765916 (T) = (0 -93.34157927489534) / Math.Sqrt((11.925590808695329 / (44)) + (1.8949430489487413 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6804762521742915 = (55.544717846577676 - 93.34157927489534) / 55.544717846577676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 168.171823399793 > 141.42523057380416. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.81367000547532 (T) = (0 -168.65982649551452) / Math.Sqrt((3.0256665704669445 / (44)) + (4.974394359634437 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2471039907207702 = (135.24118898700397 - 168.65982649551452) / 135.24118898700397 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.07502341618763 > 71.52715284606616. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -203.42713253064903 (T) = (0 -159.0495098591174) / Math.Sqrt((1.3079921367165617 / (44)) + (2.599303334509385 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3600932186383519 = (67.39119819634942 - 159.0495098591174) / 67.39119819634942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 283.346698442026 > 121.00682539428354. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -153.6489170850483 (T) = (0 -285.28923683501347) / Math.Sqrt((11.77250082672012 / (44)) + (14.101272172330201 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.4500919449505112 = (116.44021663063586 - 285.28923683501347) / 116.44021663063586 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 260.6872396516202 > 114.1374404369023. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -354.9378294929043 (T) = (0 -260.4088669440676) / Math.Sqrt((2.8457194850418803 / (44)) + (1.742486873764581 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3788144082953597 = (109.47002256080776 - 260.4088669440676) / 109.47002256080776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.6099825193455 > 153.31900611157332. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.853026521285226 (T) = (0 -163.04633277545906) / Math.Sqrt((3.2879860502441063 / (44)) + (9.761306376078682 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1092474805733087 = (146.98823809019555 - 163.04633277545906) / 146.98823809019555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.6331186509566 > 55.46005410993955. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -286.3916160677499 (T) = (0 -315.4843865319093) / Math.Sqrt((4.759110843674352 / (44)) + (10.885566974108707 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.8463903326463225 = (53.96225167694334 - 315.4843865319093) / 53.96225167694334 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int32>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.77051303676598 > 142.42038879752954. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.247980974948447 (T) = (0 -153.71977145352187) / Math.Sqrt((5.509382912958703 / (44)) + (16.539085966099183 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12348591194481433 = (136.8239421778104 - 153.71977145352187) / 136.8239421778104 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Float

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AllBitsSetBenchmark.html>) 44.71 ns 88.06 ns 1.97 0.10 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAnyBenchmark.html>) 226.14 ns 293.75 ns 1.30 0.19 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementBenchmark.html>) 103.84 ns 153.74 ns 1.48 0.07 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GetHashCodeBenchmark.html>) 770.32 ns 926.08 ns 1.20 0.01 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark.html>) 180.56 ns 418.73 ns 2.32 0.02 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAllBenchmark.html>) 77.09 ns 350.80 ns 4.55 0.04 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementOperatorBenchmark.html>) 103.23 ns 152.93 ns 1.48 0.10 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAllBenchmark.html>) 76.09 ns 361.00 ns 4.74 0.05 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAnyBenchmark.html>) 215.48 ns 301.81 ns 1.40 0.20 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAnyBenchmark.html>) 214.24 ns 304.00 ns 1.42 0.24 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SumBenchmark.html>) 71.62 ns 85.99 ns 1.20 0.15 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAllBenchmark.html>) 77.18 ns 344.19 ns 4.46 0.05 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAnyBenchmark.html>) 220.64 ns 294.51 ns 1.33 0.22 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AndNotBenchmark.html>) 131.87 ns 264.20 ns 2.00 0.06 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAllBenchmark.html>) 74.31 ns 339.85 ns 4.57 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Float*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.05613481595792 > 46.73329441946578. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.461702922443756 (T) = (0 -87.9551405562224) / Math.Sqrt((940.1918053836399 / (44)) + (1.4647023155686638 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40390145300814984 = (62.650508956850416 - 87.9551405562224) / 62.650508956850416 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 293.7467984517101 > 233.7600650839306. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.22726983346039 (T) = (0 -298.94079263040356) / Math.Sqrt((79.02632929612714 / (44)) + (35.466225102041044 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.4024323712031282 = (213.1587937990523 - 298.94079263040356) / 213.1587937990523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.74232227051573 > 109.19669864798735. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.843575558858703 (T) = (0 -154.09623013463113) / Math.Sqrt((929.7600439893577 / (44)) + (9.128150223603088 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.261143991702622 = (122.18765751450137 - 154.09623013463113) / 122.18765751450137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 926.0816362615346 > 817.8743613881471. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.979281446737087 (T) = (0 -895.7254869700346) / Math.Sqrt((83.45984627062165 / (44)) + (176.3608671139785 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1516375044783773 = (777.7842276643678 - 895.7254869700346) / 777.7842276643678 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 418.73476862262976 > 189.54075421883107. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.56070633323926 (T) = (0 -420.8491304101854) / Math.Sqrt((922.2838456381223 / (43)) + (21.35439377930078 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -1.1236385509519597 = (198.17361585451164 - 420.8491304101854) / 198.17361585451164 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 350.80226137602295 > 81.1530058157519. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -575.2937116441246 (T) = (0 -347.35506141922787) / Math.Sqrt((0.7836871671271853 / (44)) + (3.06271890334918 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.552531660828083 = (76.2993181152409 - 347.35506141922787) / 76.2993181152409 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.92880722255643 > 108.41637694498931. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.9510069437875295 (T) = (0 -154.74699026138006) / Math.Sqrt((942.1980039663678 / (44)) + (7.029956677506369 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2660183124719109 = (122.23124163128045 - 154.74699026138006) / 122.23124163128045 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.0001135083229 > 78.71757644247606. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -157.3991793800425 (T) = (0 -342.95318710208085) / Math.Sqrt((0.8739003571971207 / (44)) + (43.320701897177294 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.600507011845068 = (74.54682412592105 - 342.95318710208085) / 74.54682412592105 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 301.8149307222194 > 226.84732419878483. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.302290970751095 (T) = (0 -305.3954355218111) / Math.Sqrt((24.710410289788147 / (44)) + (42.07782806879062 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40886840612829595 = (216.76647314497367 - 305.3954355218111) / 216.76647314497367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 304.0033433014139 > 225.61243035969477. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.28200070917464 (T) = (0 -303.17396516638803) / Math.Sqrt((96.3357316216909 / (44)) + (32.333891240763016 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38304816096798583 = (219.20709178644847 - 303.17396516638803) / 219.20709178644847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 85.9888163522173 > 75.14031541403168. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.18500505662504 (T) = (0 -87.07379951492611) / Math.Sqrt((13.135876537033795 / (44)) + (6.839995258637478 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19255011778492404 = (73.01479259979398 - 87.07379951492611) / 73.01479259979398 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 344.18724745043284 > 80.0777758016306. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -359.198198724575 (T) = (0 -348.1500600278925) / Math.Sqrt((1.3018755021165964 / (44)) + (8.131371426829721 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.547287979504454 = (76.56213145001486 - 348.1500600278925) / 76.56213145001486 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.5108951493579 > 241.54002284390748. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.5660522952399 (T) = (0 -296.56324187862344) / Math.Sqrt((99.96242682143293 / (44)) + (4.387142231722718 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38589050185667906 = (213.98749863810838 - 296.56324187862344) / 213.98749863810838 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 264.1971272359376 > 139.84055887735502. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.152923781385898 (T) = (0 -265.8415301779897) / Math.Sqrt((930.5907351848957 / (44)) + (14.179969044127237 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7454019629380365 = (152.309631719732 - 265.8415301779897) / 152.309631719732 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Float.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 339.849374997873 > 78.18298329333632. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -391.8367245254965 (T) = (0 -340.6994293544408) / Math.Sqrt((2.7291090465516414 / (44)) + (5.576433429281817 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -3.5521217681849615 = (74.84409396418359 - 340.6994293544408) / 74.84409396418359 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).SortedSet(Size%3a%20512).html>) 2.65 ms 3.16 ms 1.19 0.08 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).SortedList(Size%3a%20512).html>) 2.68 ms 2.94 ms 1.10 0.04 False
[ImmutableSortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ImmutableSortedSet(Size%3a%20512).html>) 4.75 ms 5.21 ms 1.10 0.03 False
[ConcurrentStack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ConcurrentStack(Size%3a%20512).html>) 87.17 μs 100.64 μs 1.15 0.08 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).SortedDictionary(Size%3a%20512).html>) 2.99 ms 3.29 ms 1.10 0.07 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).Span(Size%3a%20512).html>) 10.74 μs 11.75 μs 1.09 0.11 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndClear(String).ImmutableSortedDictionary(Size%3a%20512).html>) 4.98 ms 5.62 ms 1.13 0.02 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndClear<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1639374991071434 > 2.7676734884895837. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.104173565882565 (T) = (0 -3025295.7367098094) / Math.Sqrt((1158325465.8808863 / (44)) + (2721983182.662541 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.154615842871718 = (2620175.147766382 - 3025295.7367098094) / 2620175.147766382 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9351961805555558 > 2.764139305408654. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/7/2022 5:17:51 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.356464429076325 (T) = (0 -2995845.1913170596) / Math.Sqrt((2655443106.487487 / (31)) + (3208822119.2639747 / (28))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (28) - 2, .025) and -0.13607334895757123 = (2637017.4021474603 - 2995845.1913170596) / 2637017.4021474603 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ImmutableSortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.210583330555556 > 4.98591023546875. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.592535553429848 (T) = (0 -5249704.537228836) / Math.Sqrt((3565287071.948271 / (44)) + (12552715242.695583 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1130007313054153 = (4716712.567719131 - 5249704.537228836) / 4716712.567719131 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ConcurrentStack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 100.64146444892471 > 92.28479813368881. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.503208667112311 (T) = (0 -102702.9913396032) / Math.Sqrt((9131901.399047429 / (44)) + (12041697.192826787 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16542247625088427 = (88125.11636981163 - 102702.9913396032) / 88125.11636981163 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2910474958333333 > 3.0845026602003203. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.38168791093312 (T) = (0 -3300918.9728733213) / Math.Sqrt((2700945276.416051 / (44)) + (2660086663.487203 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14088243882625956 = (2893303.341814349 - 3300918.9728733213) / 2893303.341814349 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.752800788118565 > 11.273809388872161. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.509153659419557 (T) = (0 -11758.553533121158) / Math.Sqrt((60116.080398446014 / (44)) + (8915.975276791232 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08369582867586176 = (10850.418744795403 - 11758.553533121158) / 10850.418744795403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.CreateAddAndClear<String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.615294444444444 > 5.259200819726563. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.631785080694417 (T) = (0 -5579363.31686399) / Math.Sqrt((3751682615.399471 / (44)) + (3343943037.054468 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.10968278839824185 = (5027890.290086823 - 5579363.31686399) / 5027890.290086823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StaticMethod4_int_string_struct_class - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.StaticMethod4_int_string_struct_class.html>) 3.61 μs 4.22 μs 1.17 0.03 False
[StaticMethod4_arrayNotCached_int_string_struct_class - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class.html>) 4.09 μs 4.63 μs 1.13 0.08 False
[Ctor4_int_string_struct_class - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.Ctor4_int_string_struct_class.html>) 3.45 μs 4.00 μs 1.16 0.05 False
[StaticMethod5_arrayNotCached_int_string_struct_class_bool - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool.html>) 5.11 μs 5.80 μs 1.13 0.03 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Reflection.Invoke.StaticMethod4_int_string_struct_class ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.224764510044643 > 3.8805545620625. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.99749304363348 (T) = (0 -4206.591870989392) / Math.Sqrt((9507.21689109116 / (44)) + (898.2494050323959 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13948409894309946 = (3691.663512365915 - 4206.591870989392) / 3691.663512365915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.StaticMethod4_arrayNotCached_int_string_struct_class ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.632389585416667 > 4.386026635703125. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/17/2022 7:01:39 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.687388824016022 (T) = (0 -4656.411868055624) / Math.Sqrt((8228.525674099697 / (44)) + (2536.3522205967665 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.12356100383686329 = (4144.33382090904 - 4656.411868055624) / 4144.33382090904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.Ctor4_int_string_struct_class ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.0024875 > 3.723293997625001. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.78843511046586 (T) = (0 -4065.4439193204366) / Math.Sqrt((7732.668445813585 / (44)) + (1167.8249280081839 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14736204660318264 = (3543.2964959546707 - 4065.4439193204366) / 3543.2964959546707 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Reflection.Invoke.StaticMethod5_arrayNotCached_int_string_struct_class_bool ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.800488891666668 > 5.431025831145834. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.80181824888511 (T) = (0 -5834.384345568784) / Math.Sqrt((9934.932255261194 / (44)) + (9136.74721623105 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12730181147112446 = (5175.53000110497 - 5834.384345568784) / 5175.53000110497 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

@vargaz this looks like https://github.com/dotnet/runtime/pull/78840?

vargaz commented 1 year ago

That PR only affects reflection, and there are bunch of tests here which don't seem to be related to reflection at all.

lewing commented 1 year ago

the diff of the changes is https://github.com/dotnet/runtime/compare/add75d0a3434658735b993db3a37a97c8b03b930...cb0c7ea3f0deb1006555ddba231f6d9721ba97c3

lewing commented 1 year ago

it looks like the v8 version changed from 11.0.162 to 11.0.216, @radical is going to pin it back and we'll see what happens 😬

lewing commented 1 year ago

this was caused by https://github.com/dotnet/runtime/pull/79254 because undid the workload versioning hack and everything after it has been testing against net7.0 wasm runtimes.

cc @SamMonoRT @radekdoulik @marek-safar @radical