dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 1922 Regressions on 12/9/2022 5:51:37 PM #10711

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Slice_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment%3a%20Multiple).html>) 168.60 ns 197.92 ns 1.17 0.12 False
[Slice_Start_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment%3a%20Multiple).html>) 291.70 ns 350.13 ns 1.20 0.13 False
[Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment%3a%20Multiple).html>) 979.13 ns 1.13 μs 1.15 0.09 False
[Slice_Start_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment%3a%20Multiple).html>) 304.12 ns 361.95 ns 1.19 0.11 False
[Slice_Repeat_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment%3a%20Single).html>) 756.18 ns 896.40 ns 1.19 0.10 False
[Slice_Repeat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat(Segment%3a%20Multiple).html>) 1.24 μs 1.46 μs 1.18 0.06 False
[Slice_StartPosition_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment%3a%20Single).html>) 126.44 ns 157.65 ns 1.25 0.12 False
[Slice_StartPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition(Segment%3a%20Multiple).html>) 173.75 ns 200.26 ns 1.15 0.15 False
[Slice_Start_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment%3a%20Single).html>) 184.41 ns 232.57 ns 1.26 0.11 False
[Slice_StartPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition(Segment%3a%20Single).html>) 137.01 ns 153.56 ns 1.12 0.09 False
[Slice_StartPosition_And_Length - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment%3a%20Single).html>) 226.88 ns 271.92 ns 1.20 0.19 False
[Slice_Repeat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Repeat(Segment%3a%20Single).html>) 865.44 ns 1.02 μs 1.18 0.07 False
[Slice_Start_And_EndPosition - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment%3a%20Single).html>) 226.07 ns 266.70 ns 1.18 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 197.92235988743397 > 177.18567282127594. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.518877405406933 (T) = (0 -202.7045277235993) / Math.Sqrt((7.74880234240451 / (44)) + (55.15055086716372 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19041669801508174 = (170.28031281952934 - 202.7045277235993) / 170.28031281952934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 350.1305290205563 > 307.44407417885117. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.23696868966313 (T) = (0 -348.4570633390689) / Math.Sqrt((47.037409026319764 / (44)) + (25.147655277881256 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18104446377011776 = (295.04144342434654 - 348.4570633390689) / 295.04144342434654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1257044070730031 > 1018.4313616942279. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.76998596056479 (T) = (0 -1132.2783371880653) / Math.Sqrt((507.9592338522709 / (44)) + (783.3442909525414 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16201903219570601 = (974.406017299524 - 1132.2783371880653) / 974.406017299524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.9522386943119 > 320.58757056165683. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.51154440305134 (T) = (0 -359.21893417920535) / Math.Sqrt((25.091528603303146 / (44)) + (134.2726660427784 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17706109648765916 = (305.1829129780194 - 359.21893417920535) / 305.1829129780194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat_StartPosition_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 896.3998395257353 > 791.1629829933098. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.129326084305122 (T) = (0 -907.5184643486697) / Math.Sqrt((749.3372694873966 / (44)) + (1165.9263934653638 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19372103424557513 = (760.2433385302758 - 907.5184643486697) / 760.2433385302758 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4632897004821581 > 1.280391183649244. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.53226989587857 (T) = (0 -1470.322082448716) / Math.Sqrt((472.1679724299574 / (44)) + (165.7637494100169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1955638150074219 = (1229.814807032771 - 1470.322082448716) / 1229.814807032771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 157.651999910994 > 133.27816282562117. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.04543256682334 (T) = (0 -155.43679116786527) / Math.Sqrt((2.533955291854354 / (44)) + (3.1965320233402688 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22586430331189639 = (126.79771386435218 - 155.43679116786527) / 126.79771386435218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 200.2626097402163 > 185.2991957050644. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 5:51:37 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.137246288330198 (T) = (0 -197.21144851002373) / Math.Sqrt((18.151887331455605 / (45)) + (32.62229025818346 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.11328053281699349 = (177.14443277922862 - 197.21144851002373) / 177.14443277922862 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_Length(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 232.56969725060424 > 194.94879794989387. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.99315150844873 (T) = (0 -221.82693819480966) / Math.Sqrt((18.517376723056536 / (44)) + (89.19905872923223 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1893129310972052 = (186.51688079281402 - 221.82693819480966) / 186.51688079281402 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.56326433206314 > 144.38586512065675. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.616097757156435 (T) = (0 -155.99142699419) / Math.Sqrt((6.059356812812471 / (43)) + (7.492576447231934 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.12127432128517877 = (139.11977116839367 - 155.99142699419) / 139.11977116839367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_StartPosition_And_Length(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 271.9177404578498 > 240.41243886641726. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.74384614382784 (T) = (0 -272.3166263147044) / Math.Sqrt((47.33209047975967 / (44)) + (3.647884122814358 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17168117569711702 = (232.41529518700662 - 272.3166263147044) / 232.41529518700662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Repeat(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0199767037539629 > 905.1480858010625. IsChangePoint: Marked as a change because one of 10/20/2022 5:12:08 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.91297953179902 (T) = (0 -1023.9921291190827) / Math.Sqrt((93.20411266984439 / (44)) + (145.52529598092596 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.18364363695718636 = (865.1186025478728 - 1023.9921291190827) / 865.1186025478728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySequence.Slice_Start_And_EndPosition(Segment: Single) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.7036457369877 > 241.66848163482152. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.157850395118523 (T) = (0 -269.43697593752916) / Math.Sqrt((19.939290877412624 / (44)) + (27.358908916864227 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16400696603130746 = (231.47367996961137 - 269.43697593752916) / 231.47367996961137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b1025%2c%202048%5d).html>) 250.09 μs 449.92 μs 1.80 0.05 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%2033%2c%20%20128%5d).html>) 345.21 μs 478.99 μs 1.39 0.03 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%209%2c%20%20%2032%5d).html>) 642.46 μs 737.98 μs 1.15 0.03 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20129%2c%201024%5d).html>) 250.32 μs 448.34 μs 1.79 0.04 False
[ReadLine - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange%3a%20%5b%20%20%200%2c%201024%5d).html>) 247.65 μs 443.93 μs 1.79 0.02 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [1025, 2048]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 449.92346938775506 > 263.4478261040707. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.93017220587247 (T) = (0 -449630.8336568549) / Math.Sqrt((243709322.8936953 / (44)) + (2590309.8868702245 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7714038731951897 = (253827.39671096476 - 449630.8336568549) / 253827.39671096476 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 478.9940641414141 > 356.33131760555193. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 10:11:20 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.60623622904413 (T) = (0 -483864.7697008473) / Math.Sqrt((264496248.83511817 / (44)) + (110852439.23642598 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.4560459655913539 = (332314.2133801607 - 483864.7697008473) / 332314.2133801607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 737.9793011363639 > 686.7939707161459. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.503492931924368 (T) = (0 -720843.0475647617) / Math.Sqrt((1839148387.745977 / (44)) + (30858818.99515411 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11817137814186991 = (644662.4029695952 - 720843.0475647617) / 644662.4029695952 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 129, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 448.3428555952382 > 264.76808312872026. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -92.23129980004123 (T) = (0 -443571.72250951367) / Math.Sqrt((178256896.18906 / (44)) + (3623104.700536639 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7568847884266237 = (252476.2724519653 - 443571.72250951367) / 252476.2724519653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 443.9308226495727 > 264.5840930953581. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/7/2022 4:10:19 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.75597907990853 (T) = (0 -441325.2574755801) / Math.Sqrt((162619496.96342447 / (43)) + (1821043.072159508 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.7641164635460395 = (250167.8696362682 - 441325.2574755801) / 250167.8696362682 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetUnicodeCategory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.GetUnicodeCategory(c%3a%20%27%d7%90%27).html>) 157.11 ns 216.61 ns 1.38 0.13 False
[Char_ToUpper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToUpper(c%3a%20%27A%27%2c%20cultureName%3a%20en-US).html>) 137.57 ns 153.74 ns 1.12 0.20 False
[Char_ToLowerInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToLowerInvariant(input%3a%20%22Hello%20World!%22).html>) 493.41 ns 909.16 ns 1.84 0.08 False
[Char_ToUpperInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToUpperInvariant(input%3a%20%22Hello%20World!%22).html>) 543.67 ns 1.02 μs 1.87 0.06 False
[Char_ToLower - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToLower(c%3a%20%27A%27%2c%20cultureName%3a%20en-US).html>) 144.69 ns 161.11 ns 1.11 0.22 False
[Char_ToUpper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToUpper(c%3a%20%27a%27%2c%20cultureName%3a%20en-US).html>) 142.05 ns 161.35 ns 1.14 0.29 False
[GetUnicodeCategory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.GetUnicodeCategory(c%3a%20%27a%27).html>) 41.52 ns 68.30 ns 1.64 0.13 False
[Char_ToLower - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToLower(c%3a%20%27a%27%2c%20cultureName%3a%20en-US).html>) 139.28 ns 153.34 ns 1.10 0.24 False
[GetUnicodeCategory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.GetUnicodeCategory(c%3a%20%27.%27).html>) 40.65 ns 66.69 ns 1.64 0.13 False
[Char_IsLower - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_IsLower(input%3a%20%22Good%20afternoon%2c%20Constable!%22).html>) 1.73 μs 2.37 μs 1.37 0.08 False
[Char_IsUpper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_IsUpper(input%3a%20%22Good%20afternoon%2c%20Constable!%22).html>) 1.67 μs 2.29 μs 1.37 0.09 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Char.GetUnicodeCategory(c: 'א') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.60844766435324 > 166.88249339835394. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.017669894090695 (T) = (0 -216.74905782075362) / Math.Sqrt((80.69749467020758 / (44)) + (59.74219894438541 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3443076137426779 = (161.2347171175383 - 216.74905782075362) / 161.2347171175383 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToUpper(c: 'A', cultureName: en-US) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.7414041412922 > 144.57958492403165. IsChangePoint: Marked as a change because one of 9/19/2022 9:36:09 PM, 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.306331985360025 (T) = (0 -154.67439833016022) / Math.Sqrt((5.166687512196455 / (44)) + (2.94533137211916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10974390961421455 = (139.37846109372242 - 154.67439833016022) / 139.37846109372242 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 909.1571217600964 > 519.4412042456303. IsChangePoint: Marked as a change because one of 11/20/2022 11:32:49 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.15202025480096 (T) = (0 -922.2865821487088) / Math.Sqrt((870.3817197092818 / (44)) + (199.1671028707166 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8383837060843589 = (501.68339672304916 - 922.2865821487088) / 501.68339672304916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToUpperInvariant(input: "Hello World!") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0165581726887365 > 569.9497175172013. IsChangePoint: Marked as a change because one of 11/20/2022 11:32:49 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.3757030504318 (T) = (0 -977.4143871632524) / Math.Sqrt((133.9014667083495 / (44)) + (298.4247107427144 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.7895768519486429 = (546.1706693953718 - 977.4143871632524) / 546.1706693953718 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToLower(c: 'A', cultureName: en-US) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.11451430860225 > 151.57270840248813. IsChangePoint: Marked as a change because one of 9/19/2022 5:43:49 PM, 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.04827188375846 (T) = (0 -160.3745017305691) / Math.Sqrt((6.350037556853667 / (43)) + (1.8278559254454614 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.11174060906910875 = (144.2553239688304 - 160.3745017305691) / 144.2553239688304 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToUpper(c: 'a', cultureName: en-US) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.3485163349827 > 152.36518359229487. IsChangePoint: Marked as a change because one of 9/22/2022 9:36:10 PM, 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.408297673598527 (T) = (0 -161.69006017341462) / Math.Sqrt((19.358660329007954 / (44)) + (30.35970324084094 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11240200831304166 = (145.35218290249017 - 161.69006017341462) / 145.35218290249017 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.GetUnicodeCategory(c: 'a') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.29645131690474 > 43.95643444292432. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.03799143591516 (T) = (0 -67.24876890467763) / Math.Sqrt((1.5088778482065806 / (44)) + (0.5434115891595871 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6109310464458328 = (41.74528081325848 - 67.24876890467763) / 41.74528081325848 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToLower(c: 'a', cultureName: en-US) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.33950747366143 > 148.93639281691225. IsChangePoint: Marked as a change because one of 9/22/2022 4:52:57 PM, 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.640926070856061 (T) = (0 -155.05981642919585) / Math.Sqrt((28.459866766330656 / (43)) + (1.4185685552802028 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.09616647199166613 = (141.4564488069607 - 155.05981642919585) / 141.4564488069607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.GetUnicodeCategory(c: '.') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 66.68650857932639 > 43.639805365217995. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.94782594482391 (T) = (0 -67.83854424424933) / Math.Sqrt((5.468894797823694 / (44)) + (6.492147934235001 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5961967164132312 = (42.500115146638954 - 67.83854424424933) / 42.500115146638954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_IsLower(input: "Good afternoon, Constable!") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3699293618289494 > 1.8146214477999518. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -74.63347660526388 (T) = (0 -2365.0783269219305) / Math.Sqrt((406.81708181609423 / (44)) + (916.7569300487722 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3599962094253142 = (1739.0330285709606 - 2365.0783269219305) / 1739.0330285709606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_IsUpper(input: "Good afternoon, Constable!") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2861799494237522 > 1.7552760612369092. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.70375763211914 (T) = (0 -2304.996533616146) / Math.Sqrt((694.6120428053223 / (43)) + (984.7519620591665 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.36824268249189995 = (1684.640132274043 - 2304.996533616146) / 1684.640132274043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterDecimal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value%3a%20123456.789).html>) 992.17 ns 1.19 μs 1.20 0.03 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%20-1.7976931348623157E%2b308).html>) 7.84 μs 9.78 μs 1.25 0.02 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%201.7976931348623157E%2b308).html>) 7.82 μs 10.12 μs 1.29 0.03 False
[FormatterDateTimeOffsetNow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDateTimeOffsetNow(value%3a%2012%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00).html>) 871.99 ns 1.33 μs 1.52 0.06 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1858194568686433 > 1040.9130341285381. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.61146248116342 (T) = (0 -1199.2924322411884) / Math.Sqrt((191.7355445142664 / (44)) + (197.0591950780169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20415034095917628 = (995.965695849807 - 1199.2924322411884) / 995.965695849807 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.775908047491923 > 8.367500322224142. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.48672431006371 (T) = (0 -9758.909538799866) / Math.Sqrt((23475.594186194903 / (44)) + (7007.306376835018 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2290210291546199 = (7940.392643657624 - 9758.909538799866) / 7940.392643657624 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.117003741054 > 8.37166715549361. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.9045595345008 (T) = (0 -9830.186961708405) / Math.Sqrt((17531.951178689134 / (44)) + (13420.556881564828 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23437848411932483 = (7963.673288360834 - 9830.186961708405) / 7963.673288360834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDateTimeOffsetNow(value: 12/30/2017 3:45:22 AM -08:00) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.325444835156092 > 916.6774350912789. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.72242689898419 (T) = (0 -1333.4993865752162) / Math.Sqrt((196.28496404068338 / (44)) + (737.4234611794966 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5253955425434919 = (874.1990843579482 - 1333.4993865752162) / 874.1990843579482 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(String%2c%20String).ImmutableSortedDictionary(Size%3a%20512).html>) 2.94 ms 3.29 ms 1.12 0.11 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(String%2c%20String).SortedDictionary(Size%3a%20512).html>) 3.10 ms 3.50 ms 1.13 0.08 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryGetValueFalse(String%2c%20String).SortedList(Size%3a%20512).html>) 2.69 ms 3.09 ms 1.15 0.11 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<String, String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.288098328333333 > 3.034257580380609. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.37537028030424 (T) = (0 -3324560.822597985) / Math.Sqrt((2435932764.8929753 / (44)) + (2273676026.6900067 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15132392011016657 = (2887598.1507270937 - 3324560.822597985) / 2887598.1507270937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueFalse<String, String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.4996115384615387 > 3.1921462435156256. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.032720928494815 (T) = (0 -3513163.5239485647) / Math.Sqrt((3486859878.0505667 / (43)) + (2712115261.130819 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.14223725067824522 = (3075686.353130661 - 3513163.5239485647) / 3075686.353130661 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.TryGetValueFalse<String, String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0922373400297616 > 2.8546003909375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.445152442715536 (T) = (0 -3098239.81518193) / Math.Sqrt((2990693947.2409554 / (43)) + (1826955965.164522 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.14547597681891664 = (2704761.931180786 - 3098239.81518193) / 2704761.931180786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetDateTimeOffset - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetDateTimeOffset.html>) 185.30 μs 222.95 μs 1.20 0.02 False
[GetUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetUInt64.html>) 22.35 μs 26.16 μs 1.17 0.15 False
[GetDecimal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetDecimal.html>) 126.56 μs 145.95 μs 1.15 0.04 False
[GetSByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetSByte.html>) 26.67 μs 31.85 μs 1.19 0.17 False
[GetSingle - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetSingle.html>) 113.07 μs 123.86 μs 1.10 0.04 False
[GetUInt16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetUInt16.html>) 24.10 μs 26.54 μs 1.10 0.16 False
[GetInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetInt64.html>) 24.00 μs 25.79 μs 1.07 0.14 False
[GetByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetByte.html>) 24.10 μs 28.50 μs 1.18 0.15 False
[GetDateTime - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetDateTime.html>) 124.75 μs 154.81 μs 1.24 0.05 False
[GetUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetUInt32.html>) 23.52 μs 25.74 μs 1.09 0.15 False
[GetBoolean - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetBoolean.html>) 4.17 μs 4.96 μs 1.19 0.05 False
[GetGuid - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetGuid.html>) 158.12 μs 216.01 μs 1.37 0.04 False
[GetInt16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetInt16.html>) 24.89 μs 27.84 μs 1.12 0.14 False
[GetInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Get.GetInt32.html>) 24.85 μs 27.01 μs 1.09 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Get.GetDateTimeOffset ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 222.94659600938968 > 197.3487900089614. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.091275021712285 (T) = (0 -225345.60399163567) / Math.Sqrt((28530617.432726894 / (44)) + (9048185.202228567 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19658311474011908 = (188324.23858879 - 225345.60399163567) / 188324.23858879 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetUInt64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.15755107382149 > 23.687874319153135. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.000664010051008 (T) = (0 -25302.263544914676) / Math.Sqrt((365334.09544574725 / (44)) + (567375.6722011373 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11341942559035476 = (22724.826748463605 - 25302.263544914676) / 22724.826748463605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetDecimal ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 145.94897292222964 > 134.58275928055136. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.860844163515948 (T) = (0 -143699.78413011832) / Math.Sqrt((24970323.32701689 / (44)) + (3675776.006108352 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10280482451642065 = (130303.91319980905 - 143699.78413011832) / 130303.91319980905 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetSByte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.846893489065614 > 28.073595467409877. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.126748927563895 (T) = (0 -31331.48073139313) / Math.Sqrt((338470.91418363655 / (44)) + (447376.74076356494 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16682320491580252 = (26851.952034716345 - 31331.48073139313) / 26851.952034716345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetSingle ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 123.85672306166056 > 117.71326503822733. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.929979964470565 (T) = (0 -124100.15455609502) / Math.Sqrt((4089747.296352202 / (44)) + (800243.2122270383 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09928647871092486 = (112891.55007311716 - 124100.15455609502) / 112891.55007311716 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetUInt16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.536986505867016 > 25.240257969833433. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.50398375712406 (T) = (0 -26586.904045762054) / Math.Sqrt((402825.6207466267 / (44)) + (216579.080206826 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09838489325035704 = (24205.453124073556 - 26586.904045762054) / 24205.453124073556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetInt64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.78921204620462 > 24.553428813575543. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.48593549809932 (T) = (0 -26033.248999668518) / Math.Sqrt((253958.3676473554 / (44)) + (233992.33877249286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10202117793626567 = (23623.18394680989 - 26033.248999668518) / 23623.18394680989 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetByte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.499365762425185 > 25.342719220094303. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.671678183380976 (T) = (0 -28582.52043575382) / Math.Sqrt((1038546.4876335607 / (44)) + (16993.47345135369 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17200729576056897 = (24387.66425699195 - 28582.52043575382) / 24387.66425699195 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetDateTime ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.810625 > 133.91939861139113. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.90514949773043 (T) = (0 -152021.47393414326) / Math.Sqrt((17377006.361148123 / (44)) + (5807594.199290966 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17744044583850127 = (129111.81577925399 - 152021.47393414326) / 129111.81577925399 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetUInt32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.74368493748413 > 24.69620820787678. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.028587364974122 (T) = (0 -25871.37394732128) / Math.Sqrt((340365.2841144623 / (44)) + (10693.136339561765 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08910589632437509 = (23754.6909209055 - 25871.37394732128) / 23754.6909209055 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetBoolean ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.960650090140995 > 4.417891555830191. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.19628603344826 (T) = (0 -4960.245522771833) / Math.Sqrt((4999.626906245718 / (44)) + (1574.2742659280843 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17593700032225415 = (4218.121822353175 - 4960.245522771833) / 4218.121822353175 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetGuid ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.01070229941288 > 167.6700941066964. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -68.33582716806181 (T) = (0 -214814.31336487207) / Math.Sqrt((16464556.222314171 / (44)) + (3994928.9652794185 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34155548421455767 = (160123.31647292225 - 214814.31336487207) / 160123.31647292225 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetInt16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.839773052376763 > 26.33564099477976. IsChangePoint: Marked as a change because one of 10/18/2022 1:35:00 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.077317954387146 (T) = (0 -27666.674633568524) / Math.Sqrt((223653.53181390234 / (44)) + (362829.66270869883 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09535456675840415 = (25258.190793365997 - 27666.674633568524) / 25258.190793365997 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Get.GetInt32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.008325188784355 > 26.04617158257368. IsChangePoint: Marked as a change because one of 10/19/2022 9:06:05 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.622217526119803 (T) = (0 -27030.99822048273) / Math.Sqrt((1528485.0464700495 / (44)) + (66616.24081903874 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.07613034866099881 = (25118.702631253454 - 27030.99822048273) / 25118.702631253454 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).UnaryNegateOperatorBenchmark.html>) 443.33 ns 484.88 ns 1.09 0.07 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).LessThanOrEqualAnyBenchmark.html>) 411.66 ns 629.50 ns 1.53 0.09 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).LessThanAnyBenchmark.html>) 392.48 ns 602.13 ns 1.53 0.07 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).ConditionalSelectBenchmark.html>) 179.91 ns 424.83 ns 2.36 0.04 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GreaterThanOrEqualAnyBenchmark.html>) 69.18 ns 593.61 ns 8.58 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).LessThanOrEqualAllBenchmark.html>) 69.33 ns 692.60 ns 9.99 0.04 False
[DotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).DotBenchmark.html>) 427.23 ns 492.91 ns 1.15 0.06 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GreaterThanOrEqualAllBenchmark.html>) 401.99 ns 679.41 ns 1.69 0.11 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).OnesComplementBenchmark.html>) 103.26 ns 154.38 ns 1.50 0.11 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).AndNotBenchmark.html>) 131.89 ns 264.07 ns 2.00 0.06 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).OnesComplementOperatorBenchmark.html>) 103.54 ns 152.55 ns 1.47 0.12 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).AllBitsSetBenchmark.html>) 45.76 ns 89.11 ns 1.95 0.02 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GreaterThanAllBenchmark.html>) 377.83 ns 655.39 ns 1.73 0.07 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).LessThanAllBenchmark.html>) 66.63 ns 646.48 ns 9.70 0.10 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).SumBenchmark.html>) 212.44 ns 301.06 ns 1.42 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).EqualsAnyBenchmark.html>) 378.36 ns 603.78 ns 1.60 0.07 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GetHashCodeBenchmark.html>) 2.43 μs 2.87 μs 1.18 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 484.87767156281114 > 473.0324070327401. IsChangePoint: Marked as a change because one of 12/2/2022 11:19:21 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.1845064964258025 (T) = (0 -491.27949736502495) / Math.Sqrt((901.9570352242414 / (43)) + (128.20708698419932 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.060827942194596786 = (463.1094994996892 - 491.27949736502495) / 463.1094994996892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 629.4958770630553 > 424.9854360425555. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -130.25887454756787 (T) = (0 -631.8798430472125) / Math.Sqrt((66.86752690863995 / (44)) + (21.929330506037008 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5527016473118499 = (406.9550928481134 - 631.8798430472125) / 406.9550928481134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 602.1271512970137 > 404.0607104461684. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -158.19723209589688 (T) = (0 -604.2887362381855) / Math.Sqrt((43.79149796361406 / (44)) + (14.634656885107741 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5810909080930362 = (382.1973380183572 - 604.2887362381855) / 382.1973380183572 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 424.8325632352387 > 193.21327761499109. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.475430230455814 (T) = (0 -424.2599806266391) / Math.Sqrt((948.489286970493 / (44)) + (56.56688099649146 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.116747686782184 = (200.43011421525932 - 424.2599806266391) / 200.43011421525932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 593.6087955048271 > 72.63364957361257. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -171.13098229000798 (T) = (0 -598.4727078411142) / Math.Sqrt((4.973890149079191 / (44)) + (141.10670728622702 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -7.494719897447824 = (70.45231803592735 - 598.4727078411142) / 70.45231803592735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 692.5955573766472 > 72.67484154587262. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -118.54529953428636 (T) = (0 -683.1009750153283) / Math.Sqrt((11.995814626161492 / (44)) + (396.522233662561 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -8.693807145016615 = (70.46777027810961 - 683.1009750153283) / 70.46777027810961 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.DotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 492.90772714300107 > 449.9220978647116. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -105.94017332698692 (T) = (0 -492.4157174214688) / Math.Sqrt((10.842514167137049 / (44)) + (1.5219117340336745 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14706515326624783 = (429.28312835528453 - 492.4157174214688) / 429.28312835528453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 679.406391778403 > 422.3861495588482. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -113.376005588329 (T) = (0 -684.5115230915618) / Math.Sqrt((73.11670017306669 / (44)) + (64.96305802541382 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6819739769533969 = (406.96915200283615 - 684.5115230915618) / 406.96915200283615 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.38294895871502 > 111.36490706843304. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.884007939094626 (T) = (0 -154.2961318453155) / Math.Sqrt((954.9348946199731 / (44)) + (4.057200628361993 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26444575398661146 = (122.02669142495239 - 154.2961318453155) / 122.02669142495239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 264.0654705144042 > 141.10871222748298. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.974867007674735 (T) = (0 -267.5610356076891) / Math.Sqrt((925.8055605903223 / (44)) + (35.064293392671644 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7644604694446971 = (151.63900820736137 - 267.5610356076891) / 151.63900820736137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.55478394435553 > 111.88560256949427. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.848225667238025 (T) = (0 -153.5073083245767) / Math.Sqrt((938.2909261406519 / (44)) + (2.114802146831812 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2605434574706165 = (121.77867205990793 - 153.5073083245767) / 121.77867205990793 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.11367233883209 > 47.15621095644154. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.532838055828034 (T) = (0 -87.2747988765294) / Math.Sqrt((901.2000523632423 / (44)) + (0.8422858031577 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40311784843825277 = (62.200619123811336 - 87.2747988765294) / 62.200619123811336 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 655.3877790006603 > 397.33891094080985. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -58.293492111135116 (T) = (0 -664.8215606474107) / Math.Sqrt((55.3837187992129 / (44)) + (312.2322743038136 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.7409781417561679 = (381.86669016808486 - 664.8215606474107) / 381.86669016808486 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 646.4756831910322 > 70.81967346856311. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -166.3932330289819 (T) = (0 -652.1376969508625) / Math.Sqrt((8.087079701424061 / (44)) + (181.9950965880944 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -8.565560640271812 = (68.17558546493429 - 652.1376969508625) / 68.17558546493429 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 301.06124013847324 > 224.47236665419473. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.208829684172564 (T) = (0 -305.0818407054527) / Math.Sqrt((15.225339438250435 / (43)) + (94.82974013425503 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.4248902605283826 = (214.10900836133249 - 305.0818407054527) / 214.10900836133249 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 603.78461262109 > 397.3663128635625. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -224.03547952759814 (T) = (0 -602.7599050582116) / Math.Sqrt((39.48060899713628 / (44)) + (1.2609001291510749 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5827790782186071 = (380.82377594768843 - 602.7599050582116) / 380.82377594768843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Byte>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.868280644662834 > 2.4924139862148946. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.62924810116204 (T) = (0 -2880.862714711261) / Math.Sqrt((839.6341766621935 / (44)) + (1072.8902227468554 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21050119088423402 = (2379.892507670215 - 2880.862714711261) / 2379.892507670215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).Span(Size%3a%20512).html>) 5.76 μs 7.42 μs 1.29 0.11 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).List(Size%3a%20512).html>) 31.31 μs 33.46 μs 1.07 0.08 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).SortedDictionary(Size%3a%20512).html>) 193.80 μs 209.08 μs 1.08 0.04 False
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).ReadOnlySpan(Size%3a%20512).html>) 5.58 μs 7.40 μs 1.33 0.04 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).ImmutableArray(Size%3a%20512).html>) 20.99 μs 22.29 μs 1.06 0.05 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).Dictionary(Size%3a%20512).html>) 69.31 μs 73.25 μs 1.06 0.03 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).HashSet(Size%3a%20512).html>) 35.74 μs 38.50 μs 1.08 0.06 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).SortedList(Size%3a%20512).html>) 87.86 μs 97.39 μs 1.11 0.07 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(String).Stack(Size%3a%20512).html>) 49.62 μs 53.44 μs 1.08 0.06 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.419527720406289 > 5.89160837436535. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.78531633730241 (T) = (0 -7405.490982155982) / Math.Sqrt((79430.11598095612 / (43)) + (78.81540117900717 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.30081874894541555 = (5692.946068127996 - 7405.490982155982) / 5692.946068127996 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.462522982449414 > 32.005477412078505. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.469620698309322 (T) = (0 -33865.132711693455) / Math.Sqrt((200032.89142912527 / (44)) + (1061315.5263206535 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10246101359089975 = (30717.75989736731 - 33865.132711693455) / 30717.75989736731 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 209.07889202302633 > 203.24466017168214. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.131512323170032 (T) = (0 -209330.5908971047) / Math.Sqrt((6283446.92192129 / (44)) + (8330467.100882618 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08267935424284767 = (193344.95488139818 - 209330.5908971047) / 193344.95488139818 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.398638290332858 > 5.855330470110164. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.420029759322446 (T) = (0 -7399.442037428471) / Math.Sqrt((43246.94058896979 / (44)) + (58.668036454073636 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3149649746120294 = (5627.1019991324265 - 7399.442037428471) / 5627.1019991324265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.28516930734383 > 21.931275208976942. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.10610282855022 (T) = (0 -22463.816364985792) / Math.Sqrt((141029.5934426726 / (44)) + (70548.45142873244 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.0742786182888095 = (20910.60548218657 - 22463.816364985792) / 20910.60548218657 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 73.24509349688473 > 70.86523653199141. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.66668058765667 (T) = (0 -74098.87370869561) / Math.Sqrt((482670.24416307046 / (44)) + (1078569.9317841588 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0965653066098713 = (67573.60757452635 - 74098.87370869561) / 67573.60757452635 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.50412806069959 > 37.276145017385126. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.563953450200696 (T) = (0 -38476.05360512063) / Math.Sqrt((2024996.8915736987 / (44)) + (116096.93886708647 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07490771134957155 = (35794.750748241495 - 38476.05360512063) / 35794.750748241495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 97.3894814453125 > 92.39656911082143. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/16/2022 9:45:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.712516963011822 (T) = (0 -96553.96861071055) / Math.Sqrt((3018372.8012417792 / (44)) + (1777320.6949868163 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09149701659098335 = (88460.13057577804 - 96553.96861071055) / 88460.13057577804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 53.44405076399115 > 52.222903836311524. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/2/2022 4:08:26 PM, 10/3/2022 2:18:04 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.995792885655261 (T) = (0 -54111.36627337299) / Math.Sqrt((235226.6292325235 / (44)) + (6387923.608694241 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07847112226069128 = (50174.144820813315 - 54111.36627337299) / 50174.144820813315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LerpBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.LerpBenchmark.html>) 169.57 ns 288.32 ns 1.70 0.12 False
[AddFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark.html>) 55.73 ns 68.54 ns 1.23 0.15 False
[UnitWBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.UnitWBenchmark.html>) 48.30 ns 65.02 ns 1.35 0.12 False
[SubtractFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmark.html>) 55.78 ns 68.90 ns 1.24 0.15 False
[MultiplyFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.MultiplyFunctionBenchmark.html>) 53.57 ns 74.29 ns 1.39 0.14 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.AbsBenchmark.html>) 43.00 ns 108.02 ns 2.51 0.18 False
[DivideOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DivideOperatorBenchmark.html>) 53.79 ns 69.06 ns 1.28 0.13 False
[CreateFromScalar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.CreateFromScalar.html>) 22.58 ns 56.05 ns 2.48 0.29 False
[TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark.html>) 133.23 ns 148.54 ns 1.11 0.09 False
[TransformVector3ByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark.html>) 133.49 ns 147.52 ns 1.11 0.08 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark.html>) 39.11 ns 57.48 ns 1.47 0.21 False
[DistanceBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DistanceBenchmark.html>) 73.84 ns 87.02 ns 1.18 0.15 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark.html>) 54.27 ns 75.18 ns 1.39 0.10 False
[SubtractOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.SubtractOperatorBenchmark.html>) 54.24 ns 67.73 ns 1.25 0.15 False
[UnitZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.UnitZBenchmark.html>) 47.01 ns 63.46 ns 1.35 0.13 False
[UnitXBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.UnitXBenchmark.html>) 50.29 ns 62.41 ns 1.24 0.15 False
[TransformVector2ByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark.html>) 106.08 ns 120.58 ns 1.14 0.07 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.OneBenchmark.html>) 23.12 ns 82.62 ns 3.57 0.19 False
[CreateFromScalarXYZWBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.CreateFromScalarXYZWBenchmark.html>) 27.42 ns 42.90 ns 1.56 0.27 False
[DivideByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark.html>) 62.74 ns 112.28 ns 1.79 0.12 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmark.html>) 53.58 ns 68.30 ns 1.27 0.15 False
[TransformByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark.html>) 203.75 ns 236.68 ns 1.16 0.11 False
[CreateFromVector2WithScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.CreateFromVector2WithScalarBenchmark.html>) 32.05 ns 65.97 ns 2.06 0.17 False
[DistanceSquaredJitOptimizeCanaryBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark.html>) 76.48 ns 89.12 ns 1.17 0.14 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.MinBenchmark.html>) 74.99 ns 93.71 ns 1.25 0.11 False
[TransformVector2ByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark.html>) 178.95 ns 210.40 ns 1.18 0.14 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.MaxBenchmark.html>) 74.92 ns 91.25 ns 1.22 0.12 False
[DivideByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark.html>) 61.50 ns 112.53 ns 1.83 0.16 False
[CreateFromVector3WithScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.CreateFromVector3WithScalarBenchmark.html>) 33.53 ns 64.54 ns 1.92 0.18 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.NegateBenchmark.html>) 53.01 ns 67.28 ns 1.27 0.15 False
[ClampBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.ClampBenchmark.html>) 186.38 ns 247.68 ns 1.33 0.08 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DivideBenchmark.html>) 53.38 ns 67.96 ns 1.27 0.19 False
[TransformVector3ByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark.html>) 206.27 ns 244.31 ns 1.18 0.12 False
[DistanceSquaredBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.DistanceSquaredBenchmark.html>) 70.85 ns 85.11 ns 1.20 0.13 False
[NegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.NegateOperatorBenchmark.html>) 55.06 ns 68.17 ns 1.24 0.15 False
[NormalizeBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark.html>) 88.95 ns 138.27 ns 1.55 0.14 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark.html>) 63.55 ns 112.19 ns 1.77 0.15 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector4.LerpBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 288.32077190126853 > 179.47927428123916. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -72.35470209475574 (T) = (0 -288.6637097572674) / Math.Sqrt((10.460507910198663 / (44)) + (35.4852692004943 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6790732374666676 = (171.9184746180542 - 288.6637097572674) / 171.9184746180542 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.AddFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.53826877242523 > 57.430373913126324. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.00385268901309 (T) = (0 -69.22161724248748) / Math.Sqrt((1.286592094664513 / (44)) + (2.092292071491246 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.271695536170958 = (54.43253929388788 - 69.22161724248748) / 54.43253929388788 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.UnitWBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.0154000360871 > 50.3641176111869. IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/23/2022 4:51:48 AM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.349246998548566 (T) = (0 -63.59318128513963) / Math.Sqrt((0.6631137286816569 / (44)) + (1.6692151244970135 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.319791588331914 = (48.18426018725814 - 63.59318128513963) / 48.18426018725814 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.SubtractFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.897725385412 > 57.09908359171796. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.133111030608198 (T) = (0 -70.10395626087174) / Math.Sqrt((1.6699363782970746 / (44)) + (13.573438854373181 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28776100593824266 = (54.43863879834992 - 70.10395626087174) / 54.43863879834992 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.MultiplyFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 74.29232337489219 > 56.22931685525518. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.66122601668116 (T) = (0 -69.03361021272663) / Math.Sqrt((1.3625240291681258 / (44)) + (3.0411918003824567 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2723529934751421 = (54.256649347110084 - 69.03361021272663) / 54.256649347110084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 108.02122121351412 > 43.87328640018271. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -112.41627641316812 (T) = (0 -108.81031255809337) / Math.Sqrt((9.383333783314033 / (44)) + (2.0879069851977072 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5863219590506523 = (42.07144906198523 - 108.81031255809337) / 42.07144906198523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DivideOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.06176749848169 > 56.67363701298207. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.187979681185695 (T) = (0 -69.37130261556406) / Math.Sqrt((7.054208452258611 / (44)) + (3.7250447006031946 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25701541455462973 = (55.18731259166209 - 69.37130261556406) / 55.18731259166209 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.CreateFromScalar ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.04729533377615 > 24.059070823069987. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -70.6673840211116 (T) = (0 -56.82444681305952) / Math.Sqrt((0.7625619026397019 / (44)) + (3.0491076695295556 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -1.5190264549799966 = (22.55809846725518 - 56.82444681305952) / 22.55809846725518 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 148.53579033662217 > 140.8710564264395. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.349315891238215 (T) = (0 -148.61990519025522) / Math.Sqrt((66.78307582828216 / (44)) + (2.501005863258947 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07864906719982169 = (137.7833715427699 - 148.61990519025522) / 137.7833715427699 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 147.5177766971835 > 142.99336813578117. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.640511803676389 (T) = (0 -148.44323464120376) / Math.Sqrt((96.59030330471992 / (44)) + (1.9569413189615192 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07321579065654961 = (138.31629755502595 - 148.44323464120376) / 138.31629755502595 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.475520542968766 > 41.826472878260915. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.25259758043677 (T) = (0 -55.27056359311476) / Math.Sqrt((2.848726242252587 / (44)) + (2.144270980234559 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3622868549050556 = (40.57189819758397 - 55.27056359311476) / 40.57189819758397 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DistanceBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.01868298862571 > 76.91322760481862. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.61279638777905 (T) = (0 -89.16760066371283) / Math.Sqrt((9.62699123304067 / (43)) + (22.832455639792805 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2078619655796534 = (73.82267444850063 - 89.16760066371283) / 73.82267444850063 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 75.17656727797984 > 56.72657689480309. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.234210821796246 (T) = (0 -68.32822031650592) / Math.Sqrt((1.4433472368014633 / (44)) + (4.922150647004542 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25674094304395145 = (54.369375562005786 - 68.32822031650592) / 54.369375562005786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.SubtractOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 67.72582729329187 > 57.56044895750467. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.68382046190081 (T) = (0 -70.59962983422984) / Math.Sqrt((8.328942739044207 / (44)) + (13.762241538832528 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.280097917882085 = (55.15174179100029 - 70.59962983422984) / 55.15174179100029 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.UnitZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.45773944199147 > 50.660899931441165. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.845188843285722 (T) = (0 -64.1510078138884) / Math.Sqrt((7.310770358405133 / (44)) + (3.8515313764706973 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3187887420599284 = (48.643884928594005 - 64.1510078138884) / 48.643884928594005 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.UnitXBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 62.40554305289698 > 50.90708850589702. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.00440093701526 (T) = (0 -63.49690306235885) / Math.Sqrt((3.963893396642676 / (44)) + (1.523093969226122 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3017465359789473 = (48.7782385490333 - 63.49690306235885) / 48.7782385490333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 120.58477221000913 > 111.05597282892192. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.04534655526582 (T) = (0 -120.69581624226878) / Math.Sqrt((2.250109488294232 / (44)) + (1.8354473366251958 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12434380092336915 = (107.34778467506749 - 120.69581624226878) / 107.34778467506749 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.61617256040131 > 23.177288436637127. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -121.89215849023667 (T) = (0 -79.30887122598001) / Math.Sqrt((0.3932631142302913 / (44)) + (3.143467061994982 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5514628750330175 = (22.331324869964376 - 79.30887122598001) / 22.331324869964376 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.CreateFromScalarXYZWBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.90089765304553 > 27.88488031645148. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.914239792298872 (T) = (0 -41.440124888294946) / Math.Sqrt((3.5391543948796396 / (44)) + (2.682480208375804 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5220417221139512 = (27.22666815646755 - 41.440124888294946) / 27.22666815646755 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DivideByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.27699265641147 > 66.18085962843156. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.6230345620906 (T) = (0 -113.6414126902885) / Math.Sqrt((2.7337658769475226 / (44)) + (4.645759628545707 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7793113203976182 = (63.868200796302105 - 113.6414126902885) / 63.868200796302105 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.29872937071458 > 57.123845829407294. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.626623667781416 (T) = (0 -68.64931245580132) / Math.Sqrt((8.426596258005086 / (44)) + (0.8139763937259353 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24923414339466576 = (54.95311893193525 - 68.64931245580132) / 54.95311893193525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformByQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 236.6822161982069 > 214.76638665421953. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.44144530903944 (T) = (0 -240.28081270235177) / Math.Sqrt((8.582378242010266 / (44)) + (35.212151552514335 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16589242610908775 = (206.09175196740637 - 240.28081270235177) / 206.09175196740637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.CreateFromVector2WithScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.97445351188902 > 33.745080035636626. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -77.87704395451141 (T) = (0 -65.97361802284313) / Math.Sqrt((0.657249481495273 / (44)) + (2.5739434855855805 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0400716257273077 = (32.3388733958411 - 65.97361802284313) / 32.3388733958411 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DistanceSquaredJitOptimizeCanaryBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.11796349096979 > 75.63512769370517. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.226190875060293 (T) = (0 -86.02782666996359) / Math.Sqrt((6.54109659170906 / (44)) + (2.8080524875949555 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2047318176963358 = (71.40827975678793 - 86.02782666996359) / 71.40827975678793 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 93.71154334471362 > 78.22203452027196. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.873505957791735 (T) = (0 -93.37897159302639) / Math.Sqrt((0.6031005076637004 / (44)) + (2.6964222108794424 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2530550314343774 = (74.52104596406679 - 93.37897159302639) / 74.52104596406679 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformVector2ByQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 210.4004051438022 > 188.458777863485. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.183932449266 (T) = (0 -212.60290987546423) / Math.Sqrt((50.756947835299705 / (44)) + (32.010402516454256 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1601248177841498 = (183.25865166951428 - 212.60290987546423) / 183.25865166951428 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.25038676979555 > 78.65840705183126. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.77802119213616 (T) = (0 -93.81045194433274) / Math.Sqrt((3.850922421811466 / (44)) + (14.366414139507361 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24031501670565492 = (75.63437568747533 - 93.81045194433274) / 75.63437568747533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DivideByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.53138692034973 > 65.62330764302992. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.4042967947668 (T) = (0 -113.9783428867578) / Math.Sqrt((4.073834465676028 / (44)) + (3.6360129181786367 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7980392197615308 = (63.39035413358472 - 113.9783428867578) / 63.39035413358472 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.CreateFromVector3WithScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 64.54458072869696 > 35.110806001634295. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -94.56416358491602 (T) = (0 -64.81867241188026) / Math.Sqrt((0.8497598284605722 / (44)) + (1.3671954412419332 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.941265383159061 = (33.38990793025913 - 64.81867241188026) / 33.38990793025913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 67.28110051225273 > 55.630272729446276. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.53067949947809 (T) = (0 -67.85208790005038) / Math.Sqrt((3.32058655653266 / (44)) + (2.343832971737925 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2650307455561393 = (53.636710521388075 - 67.85208790005038) / 53.636710521388075 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.ClampBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 247.67960926483593 > 196.3249219373842. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -101.72007508649648 (T) = (0 -247.2297648798326) / Math.Sqrt((4.074001380188185 / (43)) + (3.47787486780802 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.3175612440634804 = (187.64195288361185 - 247.2297648798326) / 187.64195288361185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 67.9591321666017 > 56.71913802906764. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.05055183607712 (T) = (0 -69.64524186040698) / Math.Sqrt((1.646837543425207 / (44)) + (7.3116795760960525 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2803727088683242 = (54.39450667607866 - 69.64524186040698) / 54.39450667607866 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.TransformVector3ByQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 244.3144375820691 > 215.2705567051507. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.309540472645732 (T) = (0 -241.2656557029944) / Math.Sqrt((12.117095879565666 / (44)) + (30.686028525618696 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16397951961481727 = (207.27654708463749 - 241.2656557029944) / 207.27654708463749 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.DistanceSquaredBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 85.10768053345306 > 74.13135997452885. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.30595964135839 (T) = (0 -85.47952010999512) / Math.Sqrt((1.3907168960389913 / (43)) + (0.8536789718839047 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1930913424191408 = (71.64541143737979 - 85.47952010999512) / 71.64541143737979 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.NegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.17175986800113 > 57.631690629231166. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.744255499194724 (T) = (0 -67.80653851444364) / Math.Sqrt((30.561882074681197 / (44)) + (1.388873361988672 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2390420372142367 = (54.72497016073358 - 67.80653851444364) / 54.72497016073358 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.NormalizeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 138.27408809271907 > 92.85758256660583. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.83574833057431 (T) = (0 -140.19277259654717) / Math.Sqrt((10.977070135711198 / (44)) + (13.63044117373849 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5613610235160057 = (89.78882557273599 - 140.19277259654717) / 89.78882557273599 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector4.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.18790695244287 > 66.5500467815181. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -109.10117103369402 (T) = (0 -113.20366090629872) / Math.Sqrt((3.7643104403971472 / (44)) + (1.8371495824068134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7843538049255656 = (63.44238490920864 - 113.20366090629872) / 63.44238490920864 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Concat_TenTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input%3a%20IEnumerable).html>) 216.70 μs 229.83 μs 1.06 0.06 False
[Concat_Once - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Concat_Once(input%3a%20IEnumerable).html>) 39.22 μs 41.56 μs 1.06 0.05 False
[Range - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Range.html>) 10.08 μs 10.92 μs 1.08 0.20 False
[Distinct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Distinct(input%3a%20IEnumerable).html>) 42.09 μs 47.43 μs 1.13 0.04 False
[Except - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Except(input%3a%20IEnumerable).html>) 61.31 μs 70.66 μs 1.15 0.03 False
[Select - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20List).html>) 16.99 μs 18.20 μs 1.07 0.09 False
[GroupBy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.GroupBy(input%3a%20IEnumerable).html>) 53.15 μs 59.19 μs 1.11 0.05 False
[Where - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Where(input%3a%20List).html>) 18.65 μs 19.74 μs 1.06 0.03 False
[EmptyTakeSelectToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray.html>) 1.15 μs 1.37 μs 1.19 0.05 False
[Append - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Append(input%3a%20IEnumerable).html>) 50.50 μs 60.25 μs 1.19 0.03 False
[CastToSameType - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.CastToSameType(input%3a%20IEnumerable).html>) 9.33 μs 9.85 μs 1.06 0.03 False
[Prepend - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Prepend(input%3a%20IEnumerable).html>) 41.17 μs 49.08 μs 1.19 0.05 False
[Skip_One - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Skip_One(input%3a%20IEnumerable).html>) 20.43 μs 21.88 μs 1.07 0.06 False
[AppendPrepend - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.AppendPrepend(input%3a%20IEnumerable).html>) 49.16 μs 59.54 μs 1.21 0.04 False
[Take_All - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Take_All(input%3a%20IEnumerable).html>) 19.71 μs 21.16 μs 1.07 0.09 False
[Select - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Select(input%3a%20IList).html>) 22.32 μs 24.17 μs 1.08 0.04 False
[Intersect - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Intersect(input%3a%20IEnumerable).html>) 71.90 μs 81.84 μs 1.14 0.02 False
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1%3a%20Array%2c%20input2%3a%20Array).html>) 253.75 ns 314.09 ns 1.24 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Concat_TenTimes(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 229.82530271796216 > 229.75652492237583. IsChangePoint: Marked as a change because one of 10/19/2022 12:02:11 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.784247992394121 (T) = (0 -232507.48557167692) / Math.Sqrt((22358804.995483108 / (44)) + (8165014.223103848 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05484837197928402 = (220417.92142638212 - 232507.48557167692) / 220417.92142638212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Concat_Once(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.55702819178853 > 41.2594198993163. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.189794956954781 (T) = (0 -41894.383687592715) / Math.Sqrt((457083.1876757722 / (44)) + (955206.4599371069 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05619239953201786 = (39665.48491179775 - 41894.383687592715) / 39665.48491179775 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Range ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.916564789920855 > 10.663289931650953. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.399831857990254 (T) = (0 -11113.916079621904) / Math.Sqrt((90700.90420764856 / (44)) + (183964.87997039352 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08659233396869871 = (10228.22978975854 - 11113.916079621904) / 10228.22978975854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Distinct(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.42765132575758 > 44.3992124047256. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/7/2022 5:17:51 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.886453869516163 (T) = (0 -46800.15639933234) / Math.Sqrt((194670.29407742477 / (31)) + (984846.6740067537 / (28))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (28) - 2, .025) and -0.1052348297112077 = (42344.083936950294 - 46800.15639933234) / 42344.083936950294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Except(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 70.66071406351003 > 64.18958028874471. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/7/2022 5:17:51 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.150031744225684 (T) = (0 -67736.14676102989) / Math.Sqrt((410099.8270688288 / (31)) + (6369692.085978561 / (28))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (28) - 2, .025) and -0.11419656823572501 = (60793.71332859759 - 67736.14676102989) / 60793.71332859759 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Select(input: List) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.19541032672112 > 18.136791537156313. IsChangePoint: Marked as a change because one of 10/11/2022 5:13:36 PM, 11/16/2022 9:45:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.680690849911762 (T) = (0 -18672.186796157242) / Math.Sqrt((144798.96255402692 / (44)) + (596033.7101400704 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.06969563474186033 = (17455.607174337238 - 18672.186796157242) / 17455.607174337238 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.18597004297784 > 56.86676063167533. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.78729537406152 (T) = (0 -59125.9230887705) / Math.Sqrt((1534620.8262405246 / (44)) + (313824.27005236637 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09057087713285941 = (54215.571246698026 - 59125.9230887705) / 54215.571246698026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Where(input: List) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.73508690326633 > 19.57589138392703. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.852617413963673 (T) = (0 -19676.154110170017) / Math.Sqrt((94207.86274640106 / (43)) + (220982.27852383515 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.06216752726366332 = (18524.52989300037 - 19676.154110170017) / 18524.52989300037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.EmptyTakeSelectToArray ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3734990318560352 > 1.2104543791874427. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.992986767250034 (T) = (0 -1378.437118493146) / Math.Sqrt((542.6301959660285 / (44)) + (635.2014380639246 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.19015753690389275 = (1158.1971930195466 - 1378.437118493146) / 1158.1971930195466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Append(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.253502455357136 > 54.107571786117646. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.709849467411775 (T) = (0 -60703.11130186693) / Math.Sqrt((2516070.151388145 / (43)) + (780918.4574663822 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.17895698162567183 = (51488.82635069771 - 60703.11130186693) / 51488.82635069771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.CastToSameType(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.850684354600968 > 9.817227983378459. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.950031693011226 (T) = (0 -9976.402364063604) / Math.Sqrt((30740.51426161337 / (44)) + (15242.748539322933 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05680719361808965 = (9440.134798769064 - 9976.402364063604) / 9440.134798769064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Prepend(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.07617300156739 > 41.899343951689104. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.61730942905983 (T) = (0 -49312.70756841132) / Math.Sqrt((742042.6846154705 / (43)) + (633384.0982416987 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.22857248843952857 = (40138.21571981142 - 49312.70756841132) / 40138.21571981142 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.88371627747253 > 21.643190378923766. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.242357678844765 (T) = (0 -22113.540460961405) / Math.Sqrt((74780.67484128462 / (44)) + (64216.492470351724 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07214889292516857 = (20625.43794698936 - 22113.540460961405) / 20625.43794698936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.AppendPrepend(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.537043381952 > 51.80040379298415. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.61594809889513 (T) = (0 -59301.9759083689) / Math.Sqrt((834372.0408966994 / (43)) + (263178.174015821 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.19550027527576114 = (49604.3180706838 - 59301.9759083689) / 49604.3180706838 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Take_All(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.161054880791713 > 20.680164903488013. IsChangePoint: Marked as a change because one of 10/6/2022 8:38:37 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.911635348182276 (T) = (0 -21158.95604022331) / Math.Sqrt((177853.18241581443 / (44)) + (106617.13331323689 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06320170950088073 = (19901.168189577467 - 21158.95604022331) / 19901.168189577467 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Select(input: IList) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.16592492905057 > 23.38012708201466. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 10:02:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.094910734813565 (T) = (0 -23709.979820886732) / Math.Sqrt((176748.58521321494 / (44)) + (70839.89782045815 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.054443001754787314 = (22485.786127300347 - 23709.979820886732) / 22485.786127300347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.Intersect(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 81.84031404263274 > 76.25651808951332. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:45:57 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.07001009053989 (T) = (0 -82054.83873938593) / Math.Sqrt((4607480.414213982 / (41)) + (2777159.1783034774 / (18))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (18) - 2, .025) and -0.11251204717045808 = (73756.35971590837 - 82054.83873938593) / 73756.35971590837 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Linq.Tests.Perf_Enumerable.SequenceEqual(input1: Array, input2: Array) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.08531208521623 > 269.8137527650159. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.80189547916939 (T) = (0 -318.6798702902642) / Math.Sqrt((26.554642911051552 / (44)) + (47.121603191397305 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.248735704260104 = (255.2020168904253 - 318.6798702902642) / 255.2020168904253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.Parse(value%3a%20%2212%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00%22).html>) 16.77 μs 18.44 μs 1.10 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22r%22).html>) 1.19 μs 1.40 μs 1.18 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20null).html>) 11.26 μs 13.85 μs 1.23 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22s%22).html>) 5.06 μs 6.02 μs 1.19 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(value%3a%2012%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00).html>) 11.22 μs 13.99 μs 1.25 0.03 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.TryParse(value%3a%20%2212%2f30%2f2017%203%3a45%3a22%20AM%20-08%3a00%22).html>) 17.46 μs 19.09 μs 1.09 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22G%22).html>) 4.97 μs 5.70 μs 1.15 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTimeOffset.ToString(format%3a%20%22o%22).html>) 1.64 μs 1.92 μs 1.17 0.04 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTimeOffset.Parse(value: "12/30/2017 3:45:22 AM -08:00") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.439708277605185 > 17.88687065387495. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.35337190437534 (T) = (0 -18669.657988236362) / Math.Sqrt((368461.9081204939 / (44)) + (35613.70657957902 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09320531447158693 = (17077.906355825347 - 18669.657988236362) / 17077.906355825347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: "r") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3958317741960635 > 1.2421755731990158. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.99245132097566 (T) = (0 -1416.2728282431208) / Math.Sqrt((954.7781532676918 / (44)) + (105.65422839268429 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2052805626732395 = (1175.0565570408878 - 1416.2728282431208) / 1175.0565570408878 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: null) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.852162970370369 > 11.907056785873984. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.85294450054554 (T) = (0 -13751.215837321099) / Math.Sqrt((19818.0869507399 / (43)) + (31482.016348351743 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.21559421488175107 = (11312.340638819815 - 13751.215837321099) / 11312.340638819815 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: "s") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.015807514027035 > 5.413072006715688. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.602564779834243 (T) = (0 -6067.132276296853) / Math.Sqrt((10704.294372271197 / (43)) + (20360.78744664127 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1755171959746525 = (5161.245022252892 - 6067.132276296853) / 5161.245022252892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(value: 12/30/2017 3:45:22 AM -08:00) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.985358535853587 > 11.981753209274148. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.98980209243506 (T) = (0 -13904.998044549464) / Math.Sqrt((32271.49145416838 / (44)) + (14517.653863122347 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2077531464629022 = (11513.112663189882 - 13904.998044549464) / 11513.112663189882 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.TryParse(value: "12/30/2017 3:45:22 AM -08:00") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.08824939537731 > 18.000257051195156. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 5:13:36 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.57649341865584 (T) = (0 -18847.924145229157) / Math.Sqrt((49844.25286364739 / (44)) + (34149.75999231042 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09712848900934756 = (17179.322507838526 - 18847.924145229157) / 17179.322507838526 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: "G") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.698579855219477 > 5.157530489662662. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/21/2022 5:53:28 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.78391990280369 (T) = (0 -5676.566857501107) / Math.Sqrt((10071.442487937104 / (43)) + (4313.90962515335 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1519764949301627 = (4927.675939989767 - 5676.566857501107) / 4927.675939989767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTimeOffset.ToString(format: "o") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9214918174197826 > 1.7396398832138416. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.574007283933877 (T) = (0 -1955.1710807166476) / Math.Sqrt((4519.448264411208 / (44)) + (1098.0324531664444 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20005319570214442 = (1629.2370102582727 - 1955.1710807166476) / 1629.2370102582727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Segment

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 18.26 ms 21.38 ms 1.17 0.04 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 1.79 ms 2.10 ms 1.18 0.03 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 18.07 ms 20.70 ms 1.15 0.03 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%208192%2c%20TestCase%3a%20Json4KB).html>) 182.87 μs 208.53 μs 1.14 0.03 False
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json40KB).html>) 1.79 ms 2.03 ms 1.14 0.04 False
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json4KB).html>) 183.06 μs 208.79 μs 1.14 0.04 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%204096%2c%20TestCase%3a%20Json4KB).html>) 215.93 μs 240.33 μs 1.11 0.02 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%204096%2c%20TestCase%3a%20Json400KB).html>) 20.63 ms 23.27 ms 1.13 0.03 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json4KB).html>) 182.47 μs 210.46 μs 1.15 0.02 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 20.31 ms 23.67 ms 1.17 0.01 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 2.07 ms 2.35 ms 1.13 0.03 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 1.79 ms 2.04 ms 1.14 0.02 False
[ReadSingleSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase%3a%20Json400KB).html>) 17.91 ms 20.47 ms 1.14 0.02 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%208192%2c%20TestCase%3a%20Json40KB).html>) 1.78 ms 2.08 ms 1.17 0.04 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json4KB).html>) 193.96 μs 217.80 μs 1.12 0.02 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%204096%2c%20TestCase%3a%20Json4KB).html>) 183.81 μs 211.87 μs 1.15 0.04 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 18.03 ms 20.67 ms 1.15 0.03 False
[ReadMultiSegmentSequenceUsingSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 1.84 ms 2.11 ms 1.15 0.01 False
[ReadSingleSegmentSequenceByN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes%3a%208192%2c%20TestCase%3a%20Json400KB).html>) 17.81 ms 20.46 ms 1.15 0.02 False
[ReadMultiSegmentSequence - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize%3a%204096%2c%20TestCase%3a%20Json40KB).html>) 2.06 ms 2.36 ms 1.14 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Segment*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.38250351048951 > 19.259369836794875. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.06060781471798 (T) = (0 -21227730.674465533) / Math.Sqrt((11193518467.233929 / (44)) + (32747349159.57076 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1530194547024959 = (18410557.24419043 - 21227730.674465533) / 18410557.24419043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1015791567708333 > 1.8809038884201388. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.67779580997722 (T) = (0 -2067226.3060084705) / Math.Sqrt((1210267493.4238036 / (44)) + (479524718.475496 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14859154166755315 = (1799792.381377997 - 2067226.3060084705) / 1799792.381377997 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.70274405357143 > 19.0859913375. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.808680708605785 (T) = (0 -20749480.164655525) / Math.Sqrt((17331285491.36362 / (44)) + (28324541442.4454 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15047370444400732 = (18035597.062762234 - 20749480.164655525) / 18035597.062762234 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.52631573464913 > 193.06590113735467. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.28558001346342 (T) = (0 -210699.87402774286) / Math.Sqrt((2329889.1129530673 / (44)) + (2660472.794292637 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14547803109855892 = (183940.56307275777 - 210699.87402774286) / 183940.56307275777 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0306145859375 > 1.8683989236111116. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.88202525958288 (T) = (0 -2051665.356983418) / Math.Sqrt((510274177.69098735 / (43)) + (398553064.41452414 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.14895881011077014 = (1785673.549764259 - 2051665.356983418) / 1785673.549764259 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.79154845238097 > 192.18594579396807. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.35021120482485 (T) = (0 -208890.3002436758) / Math.Sqrt((2776343.713532305 / (44)) + (2640393.8095128983 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13966582752316367 = (183290.8342067755 - 208890.3002436758) / 183290.8342067755 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.32739892676767 > 223.99145940503806. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.44514966688444 (T) = (0 -242086.76272072384) / Math.Sqrt((4915449.861023253 / (44)) + (10431023.078245943 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13211124614526537 = (213836.54967213425 - 242086.76272072384) / 213836.54967213425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.27070669333333 > 21.727554526363637. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.56243937346276 (T) = (0 -23455803.406821735) / Math.Sqrt((29159893965.558655 / (44)) + (30374414896.8796 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1314463112330431 = (20730814.333788183 - 23455803.406821735) / 20730814.333788183 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 210.4626419568607 > 192.03756066304723. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.05089396871999 (T) = (0 -209843.07581800074) / Math.Sqrt((8285593.296548707 / (44)) + (3203532.017271587 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14552289992568404 = (183185.4045271503 - 209843.07581800074) / 183185.4045271503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.666735707142855 > 21.428224282727275. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.31357726633999 (T) = (0 -23336263.336622708) / Math.Sqrt((80836276138.13252 / (44)) + (21558015853.4708 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13124278478598575 = (20628872.64384858 - 23336263.336622708) / 20628872.64384858 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.345444645238095 > 2.148260076484375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.24160778425263 (T) = (0 -2325651.4030886968) / Math.Sqrt((299620854.6655074 / (44)) + (304508015.7687353 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13245156927934507 = (2053643.1457008487 - 2325651.4030886968) / 2053643.1457008487 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.041664580729167 > 1.8688384553298616. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.08833653690185 (T) = (0 -2058048.2021205358) / Math.Sqrt((400705449.12129134 / (44)) + (260658119.85508984 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14975576954689873 = (1789987.2795868476 - 2058048.2021205358) / 1789987.2795868476 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequence(TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.46852220555556 > 18.62690577730769. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.19691128835365 (T) = (0 -20530323.091926407) / Math.Sqrt((22733826792.202347 / (43)) + (39540502740.14667 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15216155594873987 = (17818962.09427058 - 20530323.091926407) / 17818962.09427058 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0841953119791667 > 1.8995002604513893. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.21948721790538 (T) = (0 -2076433.4019940475) / Math.Sqrt((467835878.1887336 / (44)) + (499137748.44774157 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14724825137007458 = (1809925.0964334137 - 2076433.4019940475) / 1809925.0964334137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 217.7983806712963 > 200.96621492949697. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.94941820860763 (T) = (0 -218682.75888016913) / Math.Sqrt((5406838.752617929 / (44)) + (1619308.0597509197 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.13989452267758495 = (191844.73170946428 - 218682.75888016913) / 191844.73170946428 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 4096, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 211.86848534025094 > 193.30515368933825. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -60.27119465443106 (T) = (0 -210602.1767425456) / Math.Sqrt((4648367.985431339 / (43)) + (1292852.4703270406 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.14435616389845268 = (184035.51567817124 - 210602.1767425456) / 184035.51567817124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.67391515757576 > 19.0914593454142. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.08120018697857 (T) = (0 -20870453.53867836) / Math.Sqrt((42762947942.62497 / (44)) + (74443444445.69481 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1487553008788268 = (18167884.42474384 - 20870453.53867836) / 18167884.42474384 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequenceUsingSpan(segmentSize: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1134447869791666 > 1.9228961175520836. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.7280657370828 (T) = (0 -2103203.585159493) / Math.Sqrt((274682589.24416023 / (44)) + (267031787.7216026 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14625918530143892 = (1834841.2053129158 - 2103203.585159493) / 1834841.2053129158 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadSingleSegmentSequenceByN(numberOfBytes: 8192, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.462613101190474 > 18.976616928846152. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.256031395694976 (T) = (0 -20542194.197496757) / Math.Sqrt((79212823678.34592 / (44)) + (31448599892.726665 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13938363000801776 = (18029216.548733637 - 20542194.197496757) / 18029216.548733637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.358070743131868 > 2.166368203945313. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.143728988759165 (T) = (0 -2353269.823781833) / Math.Sqrt((433319952.7720404 / (44)) + (630875423.9005829 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13615696253639795 = (2071254.1500677057 - 2353269.823781833) / 2071254.1500677057 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 836.56 ms 1.03 secs 1.23 0.02 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 883.87 ms 1.09 secs 1.24 0.02 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 849.57 ms 1.04 secs 1.22 0.02 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 902.83 ms 1.08 secs 1.20 0.02 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0270582855 > 876.8311123875. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.41800757407945 (T) = (0 -1024527100.6446887) / Math.Sqrt((248459826992643.12 / (44)) + (94252961715091.8 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21964676092483176 = (840019531.4484433 - 1024527100.6446887) / 840019531.4484433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0945435 > 952.1455734807693. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.941031928635105 (T) = (0 -1097818648.2892797) / Math.Sqrt((178454088917029.03 / (43)) + (82171595277297.14 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2163969660450567 = (902516759.6879843 - 1097818648.2892797) / 902516759.6879843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0352347140714286 > 891.725287725. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.37522155924752 (T) = (0 -1042236721.8399637) / Math.Sqrt((156417695464446.84 / (44)) + (145462239638405.6 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22389943672166415 = (851570554.3845154 - 1042236721.8399637) / 851570554.3845154 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0808976666666668 > 938.5148426775. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -60.68660817939361 (T) = (0 -1082345929.9943469) / Math.Sqrt((175523807517463.88 / (43)) + (92235133089453.14 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.21853618388581425 = (888234542.6484033 - 1082345929.9943469) / 888234542.6484033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt16.Parse(value%3a%20%2212345%22).html>) 387.47 ns 427.67 ns 1.10 0.10 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt16.Parse(value%3a%20%2265535%22).html>) 398.60 ns 419.66 ns 1.05 0.11 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_UInt16.TryParse(value%3a%20%2265535%22).html>) 399.96 ns 425.35 ns 1.06 0.12 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt16.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 427.6669009233862 > 407.50697125450006. IsChangePoint: Marked as a change because one of 10/21/2022 10:18:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.3940231286349265 (T) = (0 -425.667848237526) / Math.Sqrt((1032.2150152440659 / (44)) + (50.2320738727824 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07021797919667018 = (397.7393918919601 - 425.667848237526) / 397.7393918919601 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.Parse(value: "65535") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 419.65879733279064 > 416.7800857174076. IsChangePoint: Marked as a change because one of 10/20/2022 8:48:07 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.803578521557649 (T) = (0 -426.57685444475095) / Math.Sqrt((88.72671790480021 / (44)) + (141.7692606037463 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08439461963323593 = (393.3778780542344 - 426.57685444475095) / 393.3778780542344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_UInt16.TryParse(value: "65535") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 425.34926812528715 > 416.3370013021359. IsChangePoint: Marked as a change because one of 10/20/2022 10:19:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.98351375966544 (T) = (0 -429.3978800076226) / Math.Sqrt((65.42506363970362 / (44)) + (64.07458992548386 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08477197424899274 = (395.8416056101583 - 429.3978800076226) / 395.8416056101583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(MyEventsListerViewModel).JsonNet.html>) 20.51 ms 22.54 ms 1.10 0.00 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(MyEventsListerViewModel).SystemTextJsonSourceGen.html>) 11.28 ms 12.51 ms 1.11 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(MyEventsListerViewModel).SystemTextJsonReflection.html>) 11.01 ms 12.27 ms 1.11 0.04 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(MyEventsListerViewModel).DataContractJsonSerializer.html>) 113.90 ms 122.59 ms 1.08 0.00 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.542514285714287 > 21.564330928863637. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.856265792441015 (T) = (0 -22450647.546820514) / Math.Sqrt((33611754535.960552 / (43)) + (12634911556.467588 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.09397440729087629 = (20522095.761286967 - 22450647.546820514) / 20522095.761286967 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.510764901754388 > 11.895674703869048. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.22326541056032 (T) = (0 -12526578.224385966) / Math.Sqrt((12870726601.077097 / (44)) + (8592549815.252623 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11006928420682129 = (11284501.249250034 - 12526578.224385966) / 11284501.249250034 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.272410006666668 > 11.599722269545456. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.230218687847916 (T) = (0 -12279600.627493387) / Math.Sqrt((17323405547.67731 / (43)) + (7547944932.167063 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.11489255749173224 = (11014156.068204312 - 12279600.627493387) / 11014156.068204312 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<MyEventsListerViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 122.59214285714285 > 120.11566869375. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.114820454326335 (T) = (0 -121854833.25153846) / Math.Sqrt((1214939482921.3665 / (43)) + (533573809631.53864 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.07611265322178612 = (113236130.8890067 - 121854833.25153846) / 113236130.8890067 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToString(Mode%3a%20SourceGen).html>) 636.66 μs 719.21 μs 1.13 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToWriter(Mode%3a%20SourceGen).html>) 594.19 μs 717.42 μs 1.21 0.05 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToString(Mode%3a%20Reflection).html>) 627.27 μs 722.96 μs 1.15 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 594.37 μs 671.51 μs 1.13 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToWriter(Mode%3a%20Reflection).html>) 590.60 μs 701.89 μs 1.19 0.05 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 647.00 μs 729.15 μs 1.13 0.04 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 586.06 μs 681.87 μs 1.16 0.05 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Hashtable).SerializeObjectProperty(Mode%3a%20Reflection).html>) 649.73 μs 734.93 μs 1.13 0.04 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Hashtable&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 719.2130097789116 > 676.8287347906652. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.18538772485229 (T) = (0 -720043.0662543806) / Math.Sqrt((49752336.31083238 / (44)) + (31676760.514037374 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12422683051949561 = (640478.4574672132 - 720043.0662543806) / 640478.4574672132 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 717.4178319493009 > 622.9440509014426. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.06493498560348 (T) = (0 -684796.4123092006) / Math.Sqrt((96631843.86515784 / (44)) + (94766601.97772348 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1469264374165046 = (597070.9105387182 - 684796.4123092006) / 597070.9105387182 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 722.9578747710623 > 673.2852109445113. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.49383974481372 (T) = (0 -721230.7426483801) / Math.Sqrt((71190400.54668608 / (44)) + (56227487.45153905 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1327807968035563 = (636690.4741707534 - 721230.7426483801) / 636690.4741707534 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 671.5106735248447 > 628.6426140925481. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.403697338458215 (T) = (0 -676803.8600898422) / Math.Sqrt((91823967.55229168 / (44)) + (36968020.697756566 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1324377915506825 = (597652.1316575576 - 676803.8600898422) / 597652.1316575576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 701.885326268116 > 624.1280043569712. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.955014129137716 (T) = (0 -689811.0073814229) / Math.Sqrt((84902678.72023867 / (44)) + (97801857.0213627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15561169840098102 = (596922.8317227265 - 689811.0073814229) / 596922.8317227265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 729.1488089285714 > 679.0138911639056. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.31613910101578 (T) = (0 -734736.429032847) / Math.Sqrt((83921586.14060013 / (44)) + (40581624.4606445 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.13289916756553058 = (648545.2987062483 - 734736.429032847) / 648545.2987062483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 681.8709239130434 > 629.8250146850962. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.197304986370014 (T) = (0 -680795.597488342) / Math.Sqrt((96313100.78401183 / (44)) + (22158221.55512095 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13829257589343744 = (598084.8965425172 - 680795.597488342) / 598084.8965425172 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Hashtable>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 734.930699829932 > 682.3978946215684. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.83004785495897 (T) = (0 -732274.2493691496) / Math.Sqrt((131233874.11298174 / (44)) + (29307117.76451329 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12580460322118694 = (650445.2435830728 - 732274.2493691496) / 650445.2435830728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BitArraySetLengthShrink - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size%3a%20512).html>) 8.94 μs 11.49 μs 1.28 0.06 False
[BitArrayLeftShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size%3a%204).html>) 141.34 ns 160.81 ns 1.14 0.09 False
[BitArrayRightShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size%3a%20512).html>) 4.67 μs 5.06 μs 1.08 0.05 False
[BitArrayBoolArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size%3a%204).html>) 213.10 ns 227.53 ns 1.07 0.11 False
[BitArraySetLengthShrink - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size%3a%204).html>) 304.61 ns 331.44 ns 1.09 0.16 False
[BitArrayCopyToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size%3a%204).html>) 369.42 ns 401.05 ns 1.09 0.09 False
[BitArraySetLengthGrow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size%3a%204).html>) 558.17 ns 631.29 ns 1.13 0.10 False
[BitArrayLengthValueCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size%3a%204).html>) 209.54 ns 236.44 ns 1.13 0.12 False
[BitArrayLeftShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size%3a%20512).html>) 4.42 μs 4.87 μs 1.10 0.10 False
[BitArrayBoolArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size%3a%20512).html>) 12.93 μs 14.63 μs 1.13 0.07 False
[BitArrayByteArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size%3a%204).html>) 235.31 ns 262.17 ns 1.11 0.17 False
[BitArrayRightShift - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size%3a%204).html>) 177.56 ns 197.93 ns 1.11 0.14 False
[BitArrayByteArrayCtor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size%3a%20512).html>) 9.43 μs 11.43 μs 1.21 0.07 False
[BitArraySetLengthGrow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size%3a%20512).html>) 9.68 μs 12.44 μs 1.29 0.03 False
[BitArraySetAll - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size%3a%204).html>) 143.91 ns 156.81 ns 1.09 0.12 False
[BitArrayCopyToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size%3a%20512).html>) 9.47 μs 11.51 μs 1.22 0.12 False
[BitArrayCopyToBoolArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size%3a%20512).html>) 150.60 μs 160.45 μs 1.07 0.09 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.490469587026096 > 9.404108920576798. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.34378579037455 (T) = (0 -11529.175885664281) / Math.Sqrt((15837.007405252818 / (44)) + (11192.354448401436 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27769772506954016 = (9023.398617256513 - 11529.175885664281) / 9023.398617256513 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 160.81059131812617 > 150.36228663481708. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.9292014625633 (T) = (0 -164.70234197921192) / Math.Sqrt((16.74691415467284 / (43)) + (50.125278528389735 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1470217425993241 = (143.5912989809344 - 164.70234197921192) / 143.5912989809344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.062920212535242 > 4.805860594950049. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.211995617212587 (T) = (0 -5074.358285742103) / Math.Sqrt((884.7670313107241 / (43)) + (3588.3412776748582 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1061529257258653 = (4587.393087996647 - 5074.358285742103) / 4587.393087996647 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 227.5328731827021 > 222.67159662147427. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.40712459129157 (T) = (0 -228.29224792625732) / Math.Sqrt((70.61178157186426 / (44)) + (3.8962000719974803 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0663804076775301 = (214.08143499509242 - 228.29224792625732) / 214.08143499509242 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 331.4367453977728 > 320.86663472207374. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.356783996032062 (T) = (0 -334.0061377516139) / Math.Sqrt((45.09207601235664 / (44)) + (45.30231826054636 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08746200720334833 = (307.14281100319573 - 334.0061377516139) / 307.14281100319573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 401.0490406862879 > 396.99545490421644. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/2/2022 9:35:40 AM, 11/11/2022 3:15:52 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.28488478790084 (T) = (0 -400.6889688826493) / Math.Sqrt((83.06579612584527 / (44)) + (11.030570301048026 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06127981936178532 = (377.55261296083904 - 400.6889688826493) / 377.55261296083904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 631.2898102303719 > 586.5122737049335. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.383945773362523 (T) = (0 -616.7744438951332) / Math.Sqrt((123.567258918834 / (44)) + (161.49453508174065 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09400128853767759 = (563.7785351419094 - 616.7744438951332) / 563.7785351419094 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 236.44086100520252 > 220.92849772208714. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.310776716124684 (T) = (0 -239.74416990459292) / Math.Sqrt((28.19902386995682 / (44)) + (77.81315606080068 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1285396165587364 = (212.4375311126839 - 239.74416990459292) / 212.4375311126839 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.870163857194957 > 4.662569012043083. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.2649981372623955 (T) = (0 -4979.380584833288) / Math.Sqrt((5206.674418300654 / (43)) + (140772.55743782356 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.11939942963551262 = (4448.260784315946 - 4979.380584833288) / 4448.260784315946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.625019875159035 > 13.578517970401009. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.7809846314612 (T) = (0 -14646.642778685477) / Math.Sqrt((19637.358127645075 / (44)) + (12962.419032412641 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1309473307086051 = (12950.773551504377 - 14646.642778685477) / 12950.773551504377 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 262.1660616075787 > 247.0529156804152. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.01696601012393 (T) = (0 -264.64535717005623) / Math.Sqrt((35.70622455853687 / (44)) + (69.8951838903958 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10787729654882337 = (238.87605422952495 - 264.64535717005623) / 238.87605422952495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 197.9299598370251 > 180.58179410427124. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.646126471781823 (T) = (0 -201.86104678200746) / Math.Sqrt((42.17308587167982 / (44)) + (56.87927477530614 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15813613226647702 = (174.29820308512834 - 201.86104678200746) / 174.29820308512834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.430062224264706 > 9.533092329663514. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.6247894593851 (T) = (0 -11468.466190058032) / Math.Sqrt((200464.4129941995 / (44)) + (6563.716671906483 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26133011718624827 = (9092.358958051103 - 11468.466190058032) / 9092.358958051103 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthGrow(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.436982621046022 > 10.16996124925674. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.18759062923894 (T) = (0 -12435.498828109274) / Math.Sqrt((37236.18019907755 / (44)) + (7516.857277273902 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.275565932483614 = (9749.00513679956 - 12435.498828109274) / 9749.00513679956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 156.81111303094596 > 151.6832740932761. IsChangePoint: Marked as a change because one of 10/6/2022 8:29:49 AM, 10/6/2022 8:38:37 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.280446147044222 (T) = (0 -157.21778151783184) / Math.Sqrt((5.466245441112411 / (44)) + (10.291023935659465 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0822963170105237 = (145.26315857018955 - 157.21778151783184) / 145.26315857018955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.514603347499385 > 9.550839731684308. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.191578650975764 (T) = (0 -11595.869289928538) / Math.Sqrt((24358.198355076354 / (44)) + (62656.95320860022 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.27424053519855146 = (9100.220068043649 - 11595.869289928538) / 9100.220068043649 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToBoolArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 160.44634885204087 > 148.0014016089933. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.89825931417857 (T) = (0 -160554.92176546808) / Math.Sqrt((72571780.11469866 / (44)) + (32277.24052950542 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12516848975182973 = (142694.11490618665 - 160554.92176546808) / 142694.11490618665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).Dictionary(Size%3a%20512).html>) 40.94 μs 47.68 μs 1.16 0.10 False
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).HashSet(Size%3a%20512).html>) 30.36 μs 32.73 μs 1.08 0.05 False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).LinkedList(Size%3a%20512).html>) 33.33 μs 35.34 μs 1.06 0.06 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).Span(Size%3a%20512).html>) 5.58 μs 7.39 μs 1.32 0.07 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).ImmutableArray(Size%3a%20512).html>) 20.71 μs 22.86 μs 1.10 0.09 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).SortedList(Size%3a%20512).html>) 75.11 μs 80.17 μs 1.07 0.07 False
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).ReadOnlySpan(Size%3a%20512).html>) 5.58 μs 7.39 μs 1.32 0.07 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).SortedDictionary(Size%3a%20512).html>) 195.11 μs 206.03 μs 1.06 0.03 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateForEach(Int32).List(Size%3a%20512).html>) 25.00 μs 27.67 μs 1.11 0.06 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateForEach<Int32>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.68074492424242 > 43.08719619584855. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.4653384678554 (T) = (0 -47416.48809889235) / Math.Sqrt((1697406.9745097891 / (44)) + (1214332.1983155478 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1459318777639827 = (41378.10372411884 - 47416.48809889235) / 41378.10372411884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.727079617774784 > 32.019016650606034. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.941415447237137 (T) = (0 -32867.15176165429) / Math.Sqrt((1850763.0940346206 / (44)) + (175356.05209231572 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06731948881375752 = (30794.10814299247 - 32867.15176165429) / 30794.10814299247 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.343881342921776 > 35.10026973809796. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.174978720313762 (T) = (0 -35604.04403320602) / Math.Sqrt((379202.40068647894 / (44)) + (168699.4119087506 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05944920073075419 = (33606.183296611256 - 35604.04403320602) / 33606.183296611256 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.387186151170682 > 5.856943992093562. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -253.8974568723738 (T) = (0 -7387.90333749562) / Math.Sqrt((2129.055331637619 / (44)) + (25.876269975516447 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3214973225317178 = (5590.554904297432 - 7387.90333749562) / 5590.554904297432 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.86143237554905 > 21.797224007121514. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.903374366224876 (T) = (0 -22647.233849131244) / Math.Sqrt((135853.1684968788 / (44)) + (112185.41519479602 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07780006323505732 = (21012.462906297038 - 22647.233849131244) / 21012.462906297038 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.16748446356276 > 77.0369388534331. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.76858456224792 (T) = (0 -80263.82089291705) / Math.Sqrt((894644.8935467227 / (44)) + (152371.1157428143 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08952252116743907 = (73668.80384162524 - 80263.82089291705) / 73668.80384162524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.388827027988615 > 5.856202842041012. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.844723773403786 (T) = (0 -7477.410950779306) / Math.Sqrt((21026.041796953767 / (44)) + (21356.060307745058 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.33309386437080335 = (5609.065610926438 - 7477.410950779306) / 5609.065610926438 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.02872817982455 > 204.57028994822826. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.50650228908808 (T) = (0 -208120.82059004792) / Math.Sqrt((8742902.661423085 / (44)) + (7088333.716826792 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.07599637602176185 = (193421.48842501186 - 208120.82059004792) / 193421.48842501186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateForEach<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.673758888994428 > 26.42477995305251. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.897768996663425 (T) = (0 -28378.34621687671) / Math.Sqrt((1636881.8992688274 / (44)) + (436299.1511497789 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10971919579897238 = (25572.546932870664 - 28378.34621687671) / 25572.546932870664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementBenchmark.html>) 115.40 ns 156.62 ns 1.36 0.12 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AndNotBenchmark.html>) 132.67 ns 263.91 ns 1.99 0.10 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ConditionalSelectBenchmark.html>) 180.03 ns 419.23 ns 2.33 0.05 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAnyBenchmark.html>) 72.72 ns 267.44 ns 3.68 0.12 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAnyBenchmark.html>) 221.19 ns 286.21 ns 1.29 0.22 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AllBitsSetBenchmark.html>) 44.08 ns 86.14 ns 1.95 0.07 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark.html>) 221.09 ns 279.42 ns 1.26 0.19 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAllBenchmark.html>) 198.27 ns 340.71 ns 1.72 0.04 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementOperatorBenchmark.html>) 101.79 ns 154.21 ns 1.51 0.06 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAllBenchmark.html>) 70.20 ns 337.93 ns 4.81 0.06 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAnyBenchmark.html>) 204.39 ns 282.46 ns 1.38 0.24 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GetHashCodeBenchmark.html>) 712.38 ns 835.47 ns 1.17 0.04 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAllBenchmark.html>) 206.80 ns 344.78 ns 1.67 0.10 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAllBenchmark.html>) 73.44 ns 329.10 ns 4.48 0.11 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SumBenchmark.html>) 66.35 ns 79.44 ns 1.20 0.12 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAnyBenchmark.html>) 72.11 ns 250.63 ns 3.48 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 156.61606106591145 > 117.90316687728789. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.949698107125962 (T) = (0 -154.88666447450854) / Math.Sqrt((955.4226920943503 / (44)) + (8.372231301005222 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26864170759694833 = (122.08857989376189 - 154.88666447450854) / 122.08857989376189 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 263.90608488948936 > 143.145216910575. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.06427116402445 (T) = (0 -265.8202801515474) / Math.Sqrt((978.740060035686 / (44)) + (3.0072773160855695 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7509592066987235 = (151.81409089063115 - 265.8202801515474) / 151.81409089063115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 419.22756740456936 > 189.7849256399215. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.67400111280954 (T) = (0 -422.04075633135943) / Math.Sqrt((944.2590595890663 / (44)) + (35.229889841424615 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1189287900060305 = (199.1764698851245 - 422.04075633135943) / 199.1764698851245 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 267.4373013926228 > 76.3973907364565. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -107.50272003011482 (T) = (0 -261.0317173998741) / Math.Sqrt((15.747557463235571 / (43)) + (36.865908327371464 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -2.487536673001254 = (74.84701721437074 - 261.0317173998741) / 74.84701721437074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 286.20677290252445 > 218.24835208036848. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.40205267812633 (T) = (0 -279.4151172807012) / Math.Sqrt((62.37760313280384 / (44)) + (33.825766975423704 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3833308059614783 = (201.98720080298864 - 279.4151172807012) / 201.98720080298864 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 86.14029490724617 > 47.494576881633506. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.502440523561926 (T) = (0 -87.78227996594643) / Math.Sqrt((922.202461851735 / (44)) + (1.4152144821403974 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4037341180159035 = (62.53483393993556 - 87.78227996594643) / 62.53483393993556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.4232086393698 > 214.34199919368632. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.752559377264724 (T) = (0 -278.1961144595855) / Math.Sqrt((32.08029528660084 / (44)) + (40.3685630203608 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39706550645144295 = (199.1288978039446 - 278.1961144595855) / 199.1288978039446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.70579507294156 > 207.4272144040256. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -89.17078440082031 (T) = (0 -342.2836123180265) / Math.Sqrt((11.900911786517877 / (44)) + (35.13176957048852 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7273768493905377 = (198.15225174448344 - 342.2836123180265) / 198.15225174448344 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.20879805245647 > 108.74869852273447. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.780485075621206 (T) = (0 -153.45094142849388) / Math.Sqrt((952.0237509688969 / (44)) + (4.717042952854703 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.261073241991475 = (121.68281454149768 - 153.45094142849388) / 121.68281454149768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 337.9259566500186 > 74.19364410800873. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -140.95650628890166 (T) = (0 -324.449482863455) / Math.Sqrt((3.2675797123483803 / (43)) + (47.152712184461926 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -3.535754038740922 = (71.5315425158104 - 324.449482863455) / 71.5315425158104 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 282.4617440962986 > 217.59113056781632. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.12157624851041 (T) = (0 -283.96238392410817) / Math.Sqrt((62.98599916995418 / (44)) + (32.84438721339782 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3677419175937148 = (207.61400982992953 - 283.96238392410817) / 207.61400982992953 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 835.4676488541148 > 749.430457754825. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.46678641928055 (T) = (0 -841.756853687016) / Math.Sqrt((274.24889099795536 / (43)) + (28.98542407016991 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.17018630264407572 = (719.3357602845272 - 841.756853687016) / 719.3357602845272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 344.7789930288287 > 213.03741305684287. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.794003462342744 (T) = (0 -349.36308932382815) / Math.Sqrt((5.172260131063701 / (44)) + (93.30850314942266 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7137141625637377 = (203.8631044521314 - 349.36308932382815) / 203.8631044521314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 329.10308864565707 > 76.82788650096154. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -181.74836841299307 (T) = (0 -328.7104882098172) / Math.Sqrt((6.203854055297968 / (44)) + (27.423420212819092 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.4622525128552524 = (73.66469899738728 - 328.7104882098172) / 73.66469899738728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 79.44398179453604 > 69.91228713107068. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.842502422067234 (T) = (0 -82.10257837673207) / Math.Sqrt((8.116780311243579 / (44)) + (18.997783167433674 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2181487249490719 = (67.39946994581027 - 82.10257837673207) / 67.39946994581027 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 250.6307210754741 > 75.21894245693065. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -105.49616888412488 (T) = (0 -255.0339709641826) / Math.Sqrt((0.6819793334880343 / (44)) + (44.98653524744409 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5487875913749254 = (71.86509882530711 - 255.0339709641826) / 71.86509882530711 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%202147483647).html>) 383.86 ns 522.61 ns 1.36 0.11 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseSpan(value%3a%20%22-2147483648%22).html>) 579.90 ns 629.98 ns 1.09 0.07 False
[ParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseHex(value%3a%20%223039%22).html>) 480.90 ns 621.74 ns 1.29 0.08 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%22-2147483648%22).html>) 494.28 ns 537.11 ns 1.09 0.07 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseSpan(value%3a%20%222147483647%22).html>) 571.64 ns 620.82 ns 1.09 0.07 False
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToStringHex(value%3a%204).html>) 491.00 ns 535.89 ns 1.09 0.08 False
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToStringHex(value%3a%202147483647).html>) 683.39 ns 746.91 ns 1.09 0.06 False
[ParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseHex(value%3a%20%227FFFFFFF%22).html>) 638.07 ns 866.51 ns 1.36 0.05 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%222147483647%22).html>) 478.40 ns 544.62 ns 1.14 0.05 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParseSpan(value%3a%20%2212345%22).html>) 340.64 ns 371.45 ns 1.09 0.11 False
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToStringHex(value%3a%2012345).html>) 593.62 ns 641.89 ns 1.08 0.06 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParse(value%3a%20%222147483647%22).html>) 480.60 ns 541.13 ns 1.13 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%20-2147483648).html>) 575.51 ns 654.81 ns 1.14 0.11 False
[ParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseHex(value%3a%20%2280000000%22).html>) 651.91 ns 860.42 ns 1.32 0.06 False
[ParseHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseHex(value%3a%20%224%22).html>) 348.06 ns 400.55 ns 1.15 0.16 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToString(value%3a%20-2147483648).html>) 574.47 ns 650.13 ns 1.13 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.Parse(value%3a%20%222147483647%22).html>) 531.17 ns 585.54 ns 1.10 0.04 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParse(value%3a%20%22-2147483648%22).html>) 486.19 ns 534.37 ns 1.10 0.07 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ParseSpan(value%3a%20%2212345%22).html>) 437.63 ns 468.82 ns 1.07 0.09 False
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.ToStringHex(value%3a%20-2147483648).html>) 696.94 ns 747.34 ns 1.07 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.Parse(value%3a%20%2212345%22).html>) 385.44 ns 409.39 ns 1.06 0.10 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryParse(value%3a%20%2212345%22).html>) 340.32 ns 364.21 ns 1.07 0.15 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int32.TryFormat(value%3a%2012345).html>) 297.60 ns 356.74 ns 1.20 0.17 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.TryFormat(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 522.6080544776755 > 406.2937298747832. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.867623933565183 (T) = (0 -520.8330048690591) / Math.Sqrt((221.21835011849316 / (44)) + (1671.1419498594162 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32603682218269175 = (392.7741644547654 - 520.8330048690591) / 392.7741644547654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseSpan(value: "-2147483648") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 629.983422843107 > 611.4646340220913. IsChangePoint: Marked as a change because one of 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.510792666431414 (T) = (0 -645.5014414915456) / Math.Sqrt((153.84627943716472 / (44)) + (1211.2001733166576 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10202212131157698 = (585.7427260382931 - 645.5014414915456) / 585.7427260382931 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseHex(value: "3039") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 621.7433412952284 > 492.2796544438542. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.30424431567453 (T) = (0 -616.2148166663209) / Math.Sqrt((67.86942534588007 / (44)) + (111.91641851220116 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2992806096154037 = (474.2738497796291 - 616.2148166663209) / 474.2738497796291 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 537.1116820748059 > 519.0134233423678. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.45945826493643 (T) = (0 -541.4434603159448) / Math.Sqrt((178.87783629598823 / (44)) + (50.75080716716833 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09047329275331598 = (496.5215231900491 - 541.4434603159448) / 496.5215231900491 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseSpan(value: "2147483647") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 620.8247911523925 > 600.8645376006299. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.484269280523403 (T) = (0 -629.9576901726313) / Math.Sqrt((51.44733985324143 / (43)) + (147.96926263657798 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.098607374726037 = (573.4147655159567 - 629.9576901726313) / 573.4147655159567 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToStringHex(value: 4) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 535.8929602218983 > 528.0881385567037. IsChangePoint: Marked as a change because one of 10/2/2022 4:42:49 PM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.076405592598396 (T) = (0 -538.0198638972757) / Math.Sqrt((811.7094291299057 / (44)) + (17.961848243865365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10041216727056199 = (488.9257679072818 - 538.0198638972757) / 488.9257679072818 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToStringHex(value: 2147483647) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 746.9082173906615 > 721.4887809426167. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.472012032816945 (T) = (0 -750.7230840568293) / Math.Sqrt((853.3009919491465 / (44)) + (66.37648806776755 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09668343394867542 = (684.5394585325367 - 750.7230840568293) / 684.5394585325367 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 866.5129142574589 > 670.9797704268356. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.38472497124035 (T) = (0 -870.9243200173241) / Math.Sqrt((71.47393453757824 / (44)) + (157.26111407197655 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36098134930760084 = (639.9237729895467 - 870.9243200173241) / 639.9237729895467 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParseSpan(value: "2147483647") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 544.6159601274663 > 505.16263952537764. IsChangePoint: Marked as a change because one of 10/11/2022 10:45:23 PM, 11/24/2022 8:21:24 PM, 11/25/2022 11:21:20 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.098655713321744 (T) = (0 -544.8503291714643) / Math.Sqrt((31.426395196152907 / (44)) + (1092.9521741795606 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12581716934189172 = (483.9598684482333 - 544.8503291714643) / 483.9598684482333 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParseSpan(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 371.4468000867416 > 361.57532888742406. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.03176352189669 (T) = (0 -375.5540821366211) / Math.Sqrt((56.90095045577613 / (44)) + (57.12168478074544 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08503878960049295 = (346.1204205196191 - 375.5540821366211) / 346.1204205196191 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToStringHex(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 641.8886395991741 > 621.5873442283065. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.430146224316285 (T) = (0 -649.1420961713123) / Math.Sqrt((842.4706547876414 / (44)) + (698.0721227698239 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11763274181947661 = (580.8187894661409 - 649.1420961713123) / 580.8187894661409 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParse(value: "2147483647") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 541.131666481598 > 509.62760373983616. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.532203310810118 (T) = (0 -532.234852863201) / Math.Sqrt((61.13447060333685 / (44)) + (120.13435232303327 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10524792899560204 = (481.5524543410557 - 532.234852863201) / 481.5524543410557 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 654.8102680540156 > 601.0539244172206. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.52230704804326 (T) = (0 -669.7454387309496) / Math.Sqrt((310.4283611135453 / (43)) + (858.9248310621808 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1602923173830421 = (577.2212990615274 - 669.7454387309496) / 577.2212990615274 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseHex(value: "80000000") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 860.4194090852811 > 672.0763018779011. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -60.62074131510144 (T) = (0 -869.6540977256217) / Math.Sqrt((76.9752597314472 / (44)) + (187.2634107156297 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35682681654516457 = (640.9470148445239 - 869.6540977256217) / 640.9470148445239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseHex(value: "4") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 400.55128236366403 > 349.62513697780815. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.54409857876498 (T) = (0 -399.54162872673425) / Math.Sqrt((59.9951384909549 / (44)) + (16.497693522930067 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18944076676645427 = (335.9071253400077 - 399.54162872673425) / 335.9071253400077 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToString(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 650.1305227229889 > 610.13548551573. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.412960988276748 (T) = (0 -653.7389789441497) / Math.Sqrt((777.6982675256738 / (44)) + (51.07714970308435 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13933347852679676 = (573.7907217379937 - 653.7389789441497) / 573.7907217379937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.Parse(value: "2147483647") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 585.5386447565385 > 554.222018360385. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.281169940809601 (T) = (0 -577.2835136463393) / Math.Sqrt((59.77295467429263 / (44)) + (135.70920007398735 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0933615543784243 = (527.98958526992 - 577.2835136463393) / 527.98958526992 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParse(value: "-2147483648") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 534.3665742431642 > 515.9056272661572. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.842256771612805 (T) = (0 -540.8642704617125) / Math.Sqrt((120.40823952069346 / (44)) + (107.27928469348976 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10145174204951604 = (491.0467248027629 - 540.8642704617125) / 491.0467248027629 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ParseSpan(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.8230715781307 > 458.8496019635215. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.935467247866006 (T) = (0 -470.6838847552359) / Math.Sqrt((59.75099855674039 / (44)) + (178.8374027388447 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07432416108792701 = (438.12091527253034 - 470.6838847552359) / 438.12091527253034 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.ToStringHex(value: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 747.3393209427886 > 721.4832585541193. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.659344096770624 (T) = (0 -757.5973079519046) / Math.Sqrt((1186.4270502452237 / (44)) + (238.69736102209708 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10148063861303386 = (687.7990237811701 - 757.5973079519046) / 687.7990237811701 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.3932398802441 > 406.1885778446321. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.484201543652166 (T) = (0 -412.831196527388) / Math.Sqrt((46.832861406569876 / (44)) + (12.757102689916632 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05848553646221165 = (390.02063070903966 - 412.831196527388) / 390.02063070903966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.2104402471232 > 363.5869805923685. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.01575796447905 (T) = (0 -370.9151007331655) / Math.Sqrt((80.48342488572442 / (44)) + (84.67931781662602 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07970869868277518 = (343.53256687259733 - 370.9151007331655) / 343.53256687259733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int32.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 356.7403350716749 > 316.6984121160493. IsChangePoint: Marked as a change because one of 10/3/2022 11:00:05 AM, 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.31300394766412 (T) = (0 -362.804370967411) / Math.Sqrt((68.13959930590782 / (44)) + (75.49186854531172 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19738780130914563 = (302.9965484621978 - 362.804370967411) / 302.9965484621978 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes%3a%20100).html>) 4.46 μs 4.82 μs 1.08 0.03 False
[ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes%3a%20100).html>) 4.43 μs 4.74 μs 1.07 0.04 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.818780609825987 > 4.694676598507919. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.797419995850912 (T) = (0 -4794.249202603621) / Math.Sqrt((4833.992924330204 / (44)) + (1024.7723498718315 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07096006663030129 = (4476.590072763759 - 4794.249202603621) / 4476.590072763759 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.736491292882745 > 4.646017967502628. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.020469572168933 (T) = (0 -4828.963732616128) / Math.Sqrt((5551.7298990018735 / (43)) + (11216.825559500749 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.07957861847464724 = (4473.0079403008585 - 4828.963732616128) / 4473.0079403008585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in BilinearTest

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Interpol_Vector - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/BilinearTest.Interpol_Vector.html>) 3.27 ms 4.13 ms 1.26 0.01 False
[Interpol_Scalar - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/BilinearTest.Interpol_Scalar.html>) 157.31 μs 215.08 μs 1.37 0.15 False
[Interpol_AVX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/BilinearTest.Interpol_AVX.html>) 3.90 ns 7.94 ns 2.04 0.71 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BilinearTest*'
### Payloads [Baseline]() [Compare]() ### Histogram #### BilinearTest.Interpol_Vector ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.13297916111111 > 3.4490253542236844. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.989222660596305 (T) = (0 -4096507.0304771946) / Math.Sqrt((3967106482.734546 / (44)) + (1251413575.3626754 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23906419861019398 = (3306129.767184036 - 4096507.0304771946) / 3306129.767184036 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### BilinearTest.Interpol_Scalar ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.08400267857138 > 166.1836710294548. IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.4062509836155 (T) = (0 -216490.99214766035) / Math.Sqrt((18016521.509787273 / (44)) + (7670352.785377604 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3574377033210896 = (159485.02949195847 - 216490.99214766035) / 159485.02949195847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### BilinearTest.Interpol_AVX ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.9435329984795935 > 3.5089021318834566. IsChangePoint: Marked as a change because one of 9/20/2022 10:35:57 PM, 9/23/2022 4:51:48 AM, 9/29/2022 2:21:18 AM, 10/10/2022 9:53:50 PM, 11/14/2022 1:59:26 PM, 12/10/2022 11:25:25 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.4217526326285945 (T) = (0 -6.61395982698728) / Math.Sqrt((1.793043504209844 / (51)) + (1.0814382934072515 / (8))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (51) + (8) - 2, .025) and -0.5113145431991657 = (4.376296024378078 - 6.61395982698728) / 4.376296024378078 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).AndNotBenchmark.html>) 132.98 ns 264.58 ns 1.99 0.06 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).SumBenchmark.html>) 136.63 ns 194.15 ns 1.42 0.07 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GreaterThanOrEqualAllBenchmark.html>) 282.05 ns 499.34 ns 1.77 0.10 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).LessThanAllBenchmark.html>) 70.77 ns 480.85 ns 6.79 0.08 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).OnesComplementOperatorBenchmark.html>) 103.65 ns 152.42 ns 1.47 0.11 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GetHashCodeBenchmark.html>) 1.30 μs 1.56 μs 1.20 0.01 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GreaterThanAnyBenchmark.html>) 70.67 ns 411.27 ns 5.82 0.07 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).OnesComplementBenchmark.html>) 107.70 ns 151.05 ns 1.40 0.10 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GreaterThanAllBenchmark.html>) 267.92 ns 484.70 ns 1.81 0.05 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).LessThanAnyBenchmark.html>) 275.15 ns 450.66 ns 1.64 0.18 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).AllBitsSetBenchmark.html>) 43.84 ns 87.16 ns 1.99 0.11 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).LessThanOrEqualAnyBenchmark.html>) 297.24 ns 453.53 ns 1.53 0.10 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).GreaterThanOrEqualAnyBenchmark.html>) 73.02 ns 415.20 ns 5.69 0.05 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).LessThanOrEqualAllBenchmark.html>) 73.86 ns 484.53 ns 6.56 0.06 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).ConditionalSelectBenchmark.html>) 185.32 ns 417.91 ns 2.26 0.09 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt16).EqualsAnyBenchmark.html>) 282.56 ns 429.53 ns 1.52 0.15 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 264.58252355303824 > 141.30798210212464. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.806173741902704 (T) = (0 -266.15565702169914) / Math.Sqrt((919.1345891918432 / (44)) + (6.116363861942842 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7547416464368115 = (151.67797354223418 - 266.15565702169914) / 151.67797354223418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 194.1472955587738 > 142.44098056975074. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.05436317771078 (T) = (0 -192.04810929456306) / Math.Sqrt((6.684209244742405 / (44)) + (27.850317131679912 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4049053290404555 = (136.69825669017223 - 192.04810929456306) / 136.69825669017223 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 499.34133510726616 > 294.9342241742089. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -142.54973465078524 (T) = (0 -501.8866883352538) / Math.Sqrt((11.598679954149073 / (44)) + (31.585880676598347 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7768095381341025 = (282.4651025130722 - 501.8866883352538) / 282.4651025130722 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 480.85361499168835 > 74.46317625727625. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -336.98716122639354 (T) = (0 -476.67939857821705) / Math.Sqrt((1.7753066554347152 / (44)) + (21.069536935159995 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -5.657938438878678 = (71.5956452517904 - 476.67939857821705) / 71.5956452517904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.41693420105835 > 108.8315559093533. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.808256945032928 (T) = (0 -154.05946899650652) / Math.Sqrt((938.8332481646713 / (44)) + (4.687571291656846 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2588489011106191 = (122.38122372000929 - 154.05946899650652) / 122.38122372000929 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5587112956020794 > 1.367887817098495. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.5117796694205 (T) = (0 -1558.0588936330132) / Math.Sqrt((196.42286872536314 / (44)) + (248.60372511245683 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19534057483194442 = (1303.443492539408 - 1558.0588936330132) / 1303.443492539408 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 411.2655056767921 > 74.40838543261204. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -182.73670986983362 (T) = (0 -407.7148193728084) / Math.Sqrt((6.417790876608835 / (44)) + (48.39281992189916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.6507018821006225 = (72.15295159426147 - 407.7148193728084) / 72.15295159426147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.05047039446848 > 110.97823794731933. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.769782655492085 (T) = (0 -154.6553983486364) / Math.Sqrt((897.5779543437988 / (44)) + (34.6603104561593 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26358788389800764 = (122.393859833116 - 154.6553983486364) / 122.393859833116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 484.69554272128056 > 282.93907014956073. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -122.64606106830816 (T) = (0 -492.45085480933045) / Math.Sqrt((4.410439893221271 / (44)) + (47.808165034787045 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8233697895058074 = (270.07733573495295 - 492.45085480933045) / 270.07733573495295 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 450.6556727952514 > 287.1244131679359. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -76.00590771106849 (T) = (0 -430.6238746884448) / Math.Sqrt((55.743065931807706 / (44)) + (44.94225586600212 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5733798868510317 = (273.69351692317423 - 430.6238746884448) / 273.69351692317423 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.15724049877691 > 46.23574618039306. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.639154936808633 (T) = (0 -88.13407763971243) / Math.Sqrt((920.0741422273163 / (44)) + (3.2450582474410052 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4166247523692327 = (62.21413080091441 - 88.13407763971243) / 62.21413080091441 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 453.5295700888813 > 310.208332898239. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.01916824475574 (T) = (0 -448.08525963479116) / Math.Sqrt((78.77397488307125 / (43)) + (105.14241967356945 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.5499894181897479 = (289.0892378853241 - 448.08525963479116) / 289.0892378853241 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 415.1966008305267 > 76.56466416694383. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -242.96663275387138 (T) = (0 -416.75105903599797) / Math.Sqrt((3.8787486064323162 / (44)) + (28.58481608487826 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.6564848643316745 = (73.67668596868737 - 416.75105903599797) / 73.67668596868737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 484.5346228028784 > 76.03476230339979. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -389.3284634102721 (T) = (0 -488.3437393942772) / Math.Sqrt((0.965673745797654 / (44)) + (16.736274362304588 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -5.682823157647775 = (73.07446686441494 - 488.3437393942772) / 73.07446686441494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.9078680859672 > 191.96048014677825. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.88205044114779 (T) = (0 -422.49687103525196) / Math.Sqrt((886.1672676370312 / (44)) + (68.1139819170919 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1175398403776104 = (199.522513333166 - 422.49687103525196) / 199.522513333166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt16>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 429.53192376253213 > 290.3868936914399. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.67473367888975 (T) = (0 -432.14392028358566) / Math.Sqrt((239.8209982559352 / (44)) + (26.96311144059955 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5608122405516789 = (276.8711758249933 - 432.14392028358566) / 276.8711758249933 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 10.13 ms 11.76 ms 1.16 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 10.79 ms 12.21 ms 1.13 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 9.86 ms 11.08 ms 1.12 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 12.85 ms 14.63 ms 1.14 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 11.01 ms 12.55 ms 1.14 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(MyEventsListerViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 12.52 ms 14.13 ms 1.13 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.757284611538461 > 10.742093481521739. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.28174848222672 (T) = (0 -11515165.184382234) / Math.Sqrt((14698746125.129744 / (44)) + (8730380808.771551 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1317235722086411 = (10174892.0559369 - 11515165.184382234) / 10174892.0559369 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.212165436090228 > 11.34063476159091. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.327343335853996 (T) = (0 -12123994.475294905) / Math.Sqrt((11371154309.853138 / (43)) + (4870259559.8468075 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.12203502916144475 = (10805361.829350192 - 12123994.475294905) / 10805361.829350192 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.079675337662339 > 10.381323752916668. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.102514518333905 (T) = (0 -11226578.94913018) / Math.Sqrt((13836578631.622156 / (44)) + (4567737068.337442 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1362310860203783 = (9880541.984158345 - 11226578.94913018) / 9880541.984158345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.628436991596642 > 13.484416971973685. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.74148014972201 (T) = (0 -14599964.186357645) / Math.Sqrt((19185682233.213963 / (44)) + (11778094927.452726 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13515192122245034 = (12861683.01652071 - 14599964.186357645) / 12861683.01652071 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.545519298245612 > 11.574571743982213. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.18736918776161 (T) = (0 -12426404.09311458) / Math.Sqrt((15397328103.805847 / (43)) + (13665616327.98815 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1229364067050037 = (11065990.931380503 - 12426404.09311458) / 11065990.931380503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.129980396078432 > 13.399113483340079. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.964696954141154 (T) = (0 -14292528.295749482) / Math.Sqrt((23074303158.741257 / (44)) + (15241966984.699085 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13377224152602968 = (12606172.361842347 - 14292528.295749482) / 12606172.361842347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateFromYawPitchRollBenchmarkBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark.html>) 386.93 ns 450.58 ns 1.16 0.17 False
[CreateFromAxisAngleBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark.html>) 202.93 ns 289.85 ns 1.43 0.11 False
[CreateOrthographicBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicBenchmark.html>) 80.90 ns 117.33 ns 1.45 0.24 False
[CreateConstrainedBillboardBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark.html>) 714.30 ns 1.00 μs 1.40 0.07 False
[CreateTranslationFromVectorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark.html>) 100.64 ns 141.66 ns 1.41 0.20 False
[TransformBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark.html>) 470.78 ns 507.20 ns 1.08 0.06 False
[CreateTranslationFromScalarXYZ - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ.html>) 62.41 ns 92.68 ns 1.49 0.26 False
[SubtractOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.SubtractOperatorBenchmark.html>) 182.61 ns 218.98 ns 1.20 0.05 False
[CreateReflectionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark.html>) 425.37 ns 539.34 ns 1.27 0.03 False
[CreateLookAtBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark.html>) 595.44 ns 798.20 ns 1.34 0.08 False
[CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark.html>) 148.36 ns 191.95 ns 1.29 0.14 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark.html>) 143.24 ns 186.56 ns 1.30 0.10 False
[CreateScaleFromVectorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark.html>) 101.31 ns 139.67 ns 1.38 0.21 False
[CreateOrthographicOffCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark.html>) 102.05 ns 135.73 ns 1.33 0.20 False
[DecomposeBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark.html>) 1.08 μs 1.61 μs 1.49 0.09 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark.html>) 201.47 ns 240.85 ns 1.20 0.08 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark.html>) 157.26 ns 196.40 ns 1.25 0.11 False
[CreateRotationZWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark.html>) 118.09 ns 153.96 ns 1.30 0.18 False
[CreateFromQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark.html>) 254.86 ns 295.50 ns 1.16 0.08 False
[CreateScaleFromScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark.html>) 58.85 ns 88.72 ns 1.51 0.26 False
[TranslationBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark.html>) 77.67 ns 92.45 ns 1.19 0.22 False
[CreateRotationXWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark.html>) 118.47 ns 154.29 ns 1.30 0.18 False
[CreateWorldBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark.html>) 481.38 ns 764.61 ns 1.59 0.06 False
[CreateRotationXBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark.html>) 75.23 ns 112.69 ns 1.50 0.26 False
[CreateScaleFromScalarWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark.html>) 99.53 ns 134.42 ns 1.35 0.14 False
[CreateBillboardBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark.html>) 583.76 ns 850.38 ns 1.46 0.05 False
[CreateShadowBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark.html>) 563.00 ns 688.32 ns 1.22 0.08 False
[CreateRotationYBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark.html>) 77.17 ns 119.89 ns 1.55 0.24 False
[CreateRotationYWithCenterBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark.html>) 119.10 ns 156.56 ns 1.31 0.15 False
[NegationOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark.html>) 141.12 ns 181.27 ns 1.28 0.10 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.SubtractBenchmark.html>) 206.15 ns 240.99 ns 1.17 0.08 False
[CreateRotationZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark.html>) 81.90 ns 119.63 ns 1.46 0.26 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark.html>) 157.83 ns 196.20 ns 1.24 0.08 False
[CreateScaleFromScalarXYZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark.html>) 101.53 ns 135.57 ns 1.34 0.20 False
[Transpose - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.Transpose.html>) 118.34 ns 153.78 ns 1.30 0.12 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark.html>) 178.71 ns 217.67 ns 1.22 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Matrix4x4.CreateFromYawPitchRollBenchmarkBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 450.58341533145307 > 410.009538790708. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.555904680956303 (T) = (0 -467.64547618292715) / Math.Sqrt((122.56318656180883 / (44)) + (226.3632082155008 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2147830594622049 = (384.9621317488226 - 467.64547618292715) / 384.9621317488226 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromAxisAngleBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 289.8509109775457 > 213.43588570087056. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.84568810742901 (T) = (0 -291.1491305664633) / Math.Sqrt((13.337569980369539 / (44)) + (31.55318446422972 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.41290269592312806 = (206.06453042135317 - 291.1491305664633) / 206.06453042135317 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 117.32559737427052 > 84.90368269444475. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.38583406505472 (T) = (0 -115.22368868625587) / Math.Sqrt((12.153635760922622 / (44)) + (9.310215484306175 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3939701038567883 = (82.65865126336566 - 115.22368868625587) / 82.65865126336566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateConstrainedBillboardBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0031659358499045 > 758.4656418286827. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.16904141066366 (T) = (0 -1007.0139052879246) / Math.Sqrt((180.02547129743806 / (44)) + (652.7754675624938 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3929272389335027 = (722.9479596213127 - 1007.0139052879246) / 722.9479596213127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromVectorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.6604792786751 > 107.14608013805473. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.066998730192836 (T) = (0 -144.46227839981935) / Math.Sqrt((9.638788397840976 / (44)) + (19.412283002242514 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40862536258716836 = (102.55550001917544 - 144.46227839981935) / 102.55550001917544 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.TransformBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 507.20140669761724 > 488.65741316969815. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.553465772052817 (T) = (0 -508.1267661328823) / Math.Sqrt((89.44854997106717 / (44)) + (29.113662547165603 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08770599689857908 = (467.1545137948353 - 508.1267661328823) / 467.1545137948353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateTranslationFromScalarXYZ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.68314628616174 > 65.93543716113585. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.429743089848664 (T) = (0 -94.25498976197849) / Math.Sqrt((7.615632540247098 / (44)) + (5.452820680389556 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.49199442062378734 = (63.17382187164712 - 94.25498976197849) / 63.17382187164712 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.SubtractOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 218.98380429779647 > 189.00186213600543. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.13074171215714 (T) = (0 -220.54651139370114) / Math.Sqrt((6.752833732706637 / (44)) + (9.860523853318314 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21978149679497644 = (180.80821194057765 - 220.54651139370114) / 180.80821194057765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateReflectionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 539.3427000988142 > 446.25808428553484. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.891739274989334 (T) = (0 -526.6883380314304) / Math.Sqrt((44.31786126176893 / (44)) + (63.7207451058139 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22954124341460513 = (428.36166810374283 - 526.6883380314304) / 428.36166810374283 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateLookAtBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 798.1978686559402 > 603.3354548751663. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.08397349063202 (T) = (0 -810.6037804038893) / Math.Sqrt((77.42233874575513 / (44)) + (482.3661153021665 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40446893191355443 = (577.1603500687349 - 810.6037804038893) / 577.1603500687349 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.94624922571631 > 157.26610845331933. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/5/2022 3:14:07 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.622796233852355 (T) = (0 -195.45323507128776) / Math.Sqrt((39.15432897800532 / (20)) + (16.611671616952563 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.3054731011026514 = (149.71831660583481 - 195.45323507128776) / 149.71831660583481 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 186.5634623181884 > 150.8166143096353. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.16947272692468 (T) = (0 -187.5495084340909) / Math.Sqrt((4.471898128922851 / (44)) + (9.815794926365557 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3030837808723347 = (143.92743673667553 - 187.5495084340909) / 143.92743673667553 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 139.66958832075514 > 107.31312351637727. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.92132651927042 (T) = (0 -144.0122463717833) / Math.Sqrt((20.44735194660841 / (44)) + (18.343796617474506 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38683085095916775 = (103.84269016814902 - 144.0122463717833) / 103.84269016814902 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateOrthographicOffCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.7302401929877 > 107.59970254418785. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.901798577515265 (T) = (0 -141.45823485321202) / Math.Sqrt((6.194228998799623 / (44)) + (43.952381149501875 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3723505942492641 = (103.07732983538064 - 141.45823485321202) / 103.07732983538064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.DecomposeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6149839053179107 > 1.153246848030982. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.867238006938063 (T) = (0 -1555.871467892477) / Math.Sqrt((8687.93554170834 / (44)) + (404.87757044194365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38496985772589853 = (1123.3973499229771 - 1555.871467892477) / 1123.3973499229771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.84621366236217 > 213.7403854598131. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.537241717226472 (T) = (0 -245.49208571568278) / Math.Sqrt((15.362253431909245 / (44)) + (43.444534905732205 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1981595518028996 = (204.89098079323819 - 245.49208571568278) / 204.89098079323819 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 196.4044903297787 > 165.45053854275727. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.3555538379466 (T) = (0 -199.95843204865827) / Math.Sqrt((5.119419407595914 / (44)) + (20.09760978482591 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27124073146823857 = (157.29391538431378 - 199.95843204865827) / 157.29391538431378 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.95531958657634 > 124.50884684476736. IsChangePoint: Marked as a change because one of 10/19/2022 1:55:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.678772803111766 (T) = (0 -156.63338738274325) / Math.Sqrt((25.196224379738588 / (44)) + (9.304294273874778 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.29025996494240264 = (121.39676626309597 - 156.63338738274325) / 121.39676626309597 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateFromQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 295.50106756443154 > 246.34745091949952. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.10323845196711 (T) = (0 -289.18249598320114) / Math.Sqrt((29.637091345063993 / (44)) + (47.886626126084145 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23621668213981994 = (233.92541142758475 - 289.18249598320114) / 233.92541142758475 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.71866673731806 > 63.01387274867471. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.404545064213686 (T) = (0 -90.94475223415355) / Math.Sqrt((5.576551382122583 / (44)) + (8.381968722920108 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4957328072389578 = (60.80280635284899 - 90.94475223415355) / 60.80280635284899 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.TranslationBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.44784602530883 > 81.86393603418566. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.67129579766913 (T) = (0 -93.7384516646452) / Math.Sqrt((6.350398060999366 / (44)) + (5.488952115228744 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1869763281131764 = (78.97246932771803 - 93.7384516646452) / 78.97246932771803 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.28522960514508 > 125.54211661175734. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.006540207313282 (T) = (0 -157.44649528497126) / Math.Sqrt((36.583310786213474 / (44)) + (16.411713218041545 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29736469607244 = (121.35870180652738 - 157.44649528497126) / 121.35870180652738 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateWorldBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 764.6084039991215 > 508.70324282430187. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -87.64774666804338 (T) = (0 -770.6745761227248) / Math.Sqrt((49.38997576468119 / (44)) + (140.237673197983 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5823422074703982 = (487.04671624399066 - 770.6745761227248) / 487.04671624399066 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationXBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.68887058155607 > 80.79907834205177. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.58140889131229 (T) = (0 -110.63686316338104) / Math.Sqrt((14.599893104751333 / (44)) + (7.928865223206472 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40831395166664064 = (78.55980055615446 - 110.63686316338104) / 78.55980055615446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 134.42063224344042 > 106.81690423539855. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 12/5/2022 3:14:07 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.14564898859814 (T) = (0 -154.20507326973188) / Math.Sqrt((6.061440611789097 / (20)) + (218.76231893373648 / (39))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (20) + (39) - 2, .025) and -0.5365628439312103 = (100.35715355136846 - 154.20507326973188) / 100.35715355136846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateBillboardBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 850.3826533168377 > 613.3981226354254. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -71.03628837803666 (T) = (0 -849.8684757237929) / Math.Sqrt((75.060675314303 / (44)) + (182.1258363836523 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.45146350948693653 = (585.525209672136 - 849.8684757237929) / 585.525209672136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateShadowBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 688.322754250327 > 594.8619063705235. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.46240099284274 (T) = (0 -692.4366097938262) / Math.Sqrt((71.51117844067268 / (44)) + (72.89756655783853 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21688681786237096 = (569.023018106307 - 692.4366097938262) / 569.023018106307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.88748685304598 > 80.90604691773505. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.03682090727123 (T) = (0 -112.38647140752178) / Math.Sqrt((3.760642877841434 / (44)) + (20.15743732601769 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4469366769771494 = (77.67200403151895 - 112.38647140752178) / 77.67200403151895 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationYWithCenterBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 156.56012547605468 > 124.93047129491133. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.09607325120903 (T) = (0 -156.33611978024177) / Math.Sqrt((5.848744064582055 / (44)) + (5.299920983891779 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.302708422891664 = (120.00852764367443 - 156.33611978024177) / 120.00852764367443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 181.26912558923974 > 149.78401738303754. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.36229736732842 (T) = (0 -184.99162384180212) / Math.Sqrt((5.4934982955545015 / (44)) + (9.982684317350328 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2946854397405907 = (142.8853821657776 - 184.99162384180212) / 142.8853821657776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.98874983817055 > 216.3010083602725. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.1823460920682 (T) = (0 -242.74157767087675) / Math.Sqrt((13.949296706581425 / (44)) + (12.795255765834666 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1922263049586747 = (203.6036083596484 - 242.74157767087675) / 203.6036083596484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateRotationZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.63163886171127 > 80.82142009705349. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.1771517977132 (T) = (0 -111.4440616872508) / Math.Sqrt((9.105838722281074 / (44)) + (9.369181666328897 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4205003499898256 = (78.45408956643271 - 111.4440616872508) / 78.45408956643271 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 196.2031257915863 > 161.05089584314268. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.34706361982623 (T) = (0 -196.15991376400788) / Math.Sqrt((7.757872955356388 / (44)) + (3.60430591596059 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.268598454914126 = (154.6272683875264 - 196.15991376400788) / 154.6272683875264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromScalarXYZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 135.5710133231214 > 106.69304054512095. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.86264442103733 (T) = (0 -140.305775607537) / Math.Sqrt((9.41967388238998 / (44)) + (32.0698775717429 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3475112727370432 = (104.1221535182783 - 140.305775607537) / 104.1221535182783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.Transpose ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 153.77514335005677 > 123.50889306785075. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.98196552410468 (T) = (0 -156.54207424575833) / Math.Sqrt((5.337863347875132 / (43)) + (10.066020506921591 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.35841827526077197 = (115.23849251491214 - 156.54207424575833) / 115.23849251491214 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Matrix4x4.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 217.67017930846603 > 187.92413465086398. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.440607757893716 (T) = (0 -222.12018325399066) / Math.Sqrt((14.858137736601151 / (44)) + (10.926627416565388 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.21690112961569805 = (182.52935908124027 - 222.12018325399066) / 182.52935908124027 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Parse(numberString%3a%20-2147483648).html>) 4.47 μs 5.15 μs 1.15 0.05 False
[GreatestCommonDivisor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments%3a%201024%2c1024%20bits).html>) 101.03 μs 113.81 μs 1.13 0.03 False
[ToStringX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString%3a%20-2147483648).html>) 1.39 μs 1.52 μs 1.09 0.13 False
[ToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString%3a%201234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456.html>) 1.24 μs 1.77 μs 1.42 0.04 False
[Subtract - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Subtract(arguments%3a%201024%2c1024%20bits).html>) 1.60 μs 1.92 μs 1.20 0.06 False
[Remainder - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Remainder(arguments%3a%2065536%2c32768%20bits).html>) 35.80 ms 42.59 ms 1.19 0.15 False
[Multiply - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Multiply(arguments%3a%201024%2c1024%20bits).html>) 34.86 μs 40.38 μs 1.16 0.09 False
[GreatestCommonDivisor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments%3a%2065536%2c65536%20bits).html>) 124.25 ms 149.72 ms 1.21 0.11 False
[Remainder - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Remainder(arguments%3a%201024%2c512%20bits).html>) 14.54 μs 17.06 μs 1.17 0.06 False
[Ctor_ByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString%3a%201234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123.html>) 2.11 μs 2.42 μs 1.14 0.12 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Parse(numberString%3a%201234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012.html>) 39.60 μs 46.73 μs 1.18 0.01 False
[ModPow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ModPow(arguments%3a%201024%2c1024%2c64%20bits).html>) 2.53 ms 3.02 ms 1.19 0.03 False
[Ctor_ByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString%3a%20-2147483648).html>) 262.62 ns 305.93 ns 1.16 0.16 False
[ToStringX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString%3a%20123).html>) 1.18 μs 1.27 μs 1.07 0.16 False
[ModPow - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ModPow(arguments%3a%2016384%2c16384%2c64%20bits).html>) 39.66 ms 47.14 ms 1.19 0.02 False
[Add - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Add(arguments%3a%201024%2c1024%20bits).html>) 1.54 μs 1.83 μs 1.19 0.05 False
[Add - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Add(arguments%3a%2065536%2c65536%20bits).html>) 56.65 μs 65.70 μs 1.16 0.11 False
[ToByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString%3a%20123).html>) 242.73 ns 260.54 ns 1.07 0.12 False
[Ctor_ByteArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString%3a%20123).html>) 166.26 ns 179.78 ns 1.08 0.14 False
[Divide - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Divide(arguments%3a%201024%2c512%20bits).html>) 14.56 μs 17.15 μs 1.18 0.03 False
[Subtract - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Subtract(arguments%3a%2065536%2c65536%20bits).html>) 56.69 μs 63.58 μs 1.12 0.08 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Parse(numberString%3a%20123).html>) 3.10 μs 3.30 μs 1.06 0.09 False
[ToStringX - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString%3a%20123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678.html>) 7.30 μs 8.83 μs 1.21 0.05 False
[Divide - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Divide(arguments%3a%2065536%2c32768%20bits).html>) 35.84 ms 42.51 ms 1.19 0.09 False
[Multiply - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_BigInteger.Multiply(arguments%3a%2065536%2c65536%20bits).html>) 30.47 ms 35.44 ms 1.16 0.05 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.147862286908308 > 4.695310663792764. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/19/2022 4:55:41 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.109997361311112 (T) = (0 -5209.187576356492) / Math.Sqrt((8225.930879315923 / (43)) + (8726.242632054375 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15476557084810727 = (4511.034713765021 - 5209.187576356492) / 4511.034713765021 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 1024,1024 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 113.80807549470457 > 106.5995250815616. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.0831766422316 (T) = (0 -113309.14466428202) / Math.Sqrt((1233213.9889406888 / (44)) + (1147700.5549812336 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11478176639207624 = (101642.44525724548 - 113309.14466428202) / 101642.44525724548 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.519561953371301 > 1.457392016019554. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.47919562496174 (T) = (0 -1529.5040425577051) / Math.Sqrt((1201.2273065084335 / (44)) + (1796.0823584053194 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11964995535726053 = (1366.0555562382597 - 1529.5040425577051) / 1366.0555562382597 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7673955199136746 > 1.3048489316997158. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.83989502259445 (T) = (0 -1774.5202486695098) / Math.Sqrt((333.66997607091133 / (44)) + (1925.4279450284873 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4176802910387348 = (1251.7069327170502 - 1774.5202486695098) / 1251.7069327170502 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9229671973958335 > 1.7251912314686484. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/24/2022 9:01:44 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.5114348601886 (T) = (0 -1902.23221671729) / Math.Sqrt((1829.7510249548545 / (44)) + (363.787664915278 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17262148988421075 = (1622.2048061775874 - 1902.23221671729) / 1622.2048061775874 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 65536,32768 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.587666641025635 > 37.48489643321429. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.751008117884865 (T) = (0 -42618288.80619861) / Math.Sqrt((1777272756542.771 / (44)) + (317062829804.601 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18441276557225836 = (35982632.10681223 - 42618288.80619861) / 35982632.10681223 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 1024,1024 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.37840006476684 > 36.63748950817954. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -80.71082590170958 (T) = (0 -40502.875969916655) / Math.Sqrt((148619.1946639918 / (44)) + (20207.59385031717 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15871502688889572 = (34954.993272733576 - 40502.875969916655) / 34954.993272733576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 65536,65536 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 149.7203569642857 > 130.32007555134618. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.165464552272024 (T) = (0 -150403693.4023077) / Math.Sqrt((33894016109484.918 / (44)) + (5258859785255.912 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19483965790785812 = (125877721.25480145 - 150403693.4023077) / 125877721.25480145 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 1024,512 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.062736180508903 > 15.382572234027325. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.766847203755116 (T) = (0 -17119.384241048017) / Math.Sqrt((181806.42536721856 / (44)) + (29413.741716395187 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16317395857701208 = (14717.819389621907 - 17119.384241048017) / 14717.819389621907 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4157813632854457 > 2.201653948831601. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.68499036842095 (T) = (0 -2446.3317098111897) / Math.Sqrt((4495.9910003617815 / (44)) + (3729.138722951889 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15660210563589572 = (2115.102244661923 - 2446.3317098111897) / 2115.102244661923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 46.727314457623216 > 42.21074627927435. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -70.58487221743738 (T) = (0 -46642.77314962587) / Math.Sqrt((155708.70064755267 / (43)) + (73577.90236840826 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16243382731269806 = (40125.09964326669 - 46642.77314962587) / 40125.09964326669 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.017664571527778 > 2.662116867031251. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.62947907420744 (T) = (0 -2997010.694620726) / Math.Sqrt((3404792206.8200946 / (44)) + (851422812.9958255 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1808453587307417 = (2538021.318762798 - 2997010.694620726) / 2538021.318762798 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 305.9258447022823 > 276.79908768237306. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 11/15/2022 9:22:46 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.33451978738674 (T) = (0 -290.958050806892) / Math.Sqrt((39.9877406646467 / (44)) + (55.1770734481751 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09103061819025203 = (266.68183821414556 - 290.958050806892) / 266.68183821414556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.268932703957708 > 1.2431950685432418. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.448827855749796 (T) = (0 -1287.555518657034) / Math.Sqrt((2516.3706496919153 / (44)) + (1341.2738012001967 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09752253623929383 = (1173.1472258136002 - 1287.555518657034) / 1173.1472258136002 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16384,16384,64 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.136733299999996 > 41.951553711217954. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.763732147930675 (T) = (0 -47639512.20675824) / Math.Sqrt((948092970689.8312 / (44)) + (118210188025.65366 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18201811935472925 = (40303538.01409147 - 47639512.20675824) / 40303538.01409147 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8331852208313653 > 1.6317513334273726. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.618575000578396 (T) = (0 -1825.744837957735) / Math.Sqrt((1562.886329850486 / (44)) + (173.40733710821686 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16465755128027063 = (1567.6237499604517 - 1825.744837957735) / 1567.6237499604517 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Add(arguments: 65536,65536 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 65.70472172619048 > 59.42210433228343. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.92014639944676 (T) = (0 -63781.51028124909) / Math.Sqrt((2394040.1008643773 / (44)) + (620421.9620171245 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11901528442919597 = (56997.89017071712 - 63781.51028124909) / 56997.89017071712 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 260.54027660025434 > 253.67297863914692. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.084308782924836 (T) = (0 -266.642758544928) / Math.Sqrt((37.491981238755905 / (44)) + (21.852580124278198 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09404509460593061 = (243.72190859369587 - 266.642758544928) / 243.72190859369587 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 179.7759938950942 > 174.69872093789513. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.118431784790708 (T) = (0 -183.93803641660136) / Math.Sqrt((28.587166230359994 / (44)) + (41.72381595483787 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0890374045475041 = (168.899649955575 - 183.93803641660136) / 168.899649955575 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 1024,512 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.152551955641986 > 15.319773932060118. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.27512300692224 (T) = (0 -17109.956837336133) / Math.Sqrt((43916.52469616321 / (44)) + (38593.5975760419 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16843825008833926 = (14643.441222540037 - 17109.956837336133) / 14643.441222540037 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 65536,65536 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.57967843183239 > 59.57015581647728. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.606754139004217 (T) = (0 -64024.15977072052) / Math.Sqrt((1532035.006584237 / (44)) + (2046781.925699407 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.12404686469335435 = (56958.62137223846 - 64024.15977072052) / 56958.62137223846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.304115283390047 > 3.1327093945063393. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.310415061585472 (T) = (0 -3409.9348716337013) / Math.Sqrt((5453.7776413355 / (43)) + (9946.044026719317 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.13821732488198785 = (2995.855709705744 - 3409.9348716337013) / 2995.855709705744 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.832046145594967 > 7.71549598753335. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.615322473883175 (T) = (0 -8887.097468733826) / Math.Sqrt((35861.67884997476 / (44)) + (13052.602944373228 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21029170006123238 = (7342.938457137399 - 8887.097468733826) / 7342.938457137399 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 42.50856415384615 > 37.502673597527476. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.016868071075532 (T) = (0 -42637230.02814611) / Math.Sqrt((3924741743622.694 / (44)) + (439472300713.57245 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18138177266904748 = (36090983.46914356 - 42637230.02814611) / 36090983.46914356 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 65536,65536 bits) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.44237756122449 > 31.977465620625004. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.501577923662797 (T) = (0 -35391515.544423506) / Math.Sqrt((815905418750.3031 / (44)) + (203508035919.30972 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14826302519460943 = (30821784.52835342 - 35391515.544423506) / 30821784.52835342 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).DivideBenchmark.html>) 146.92 ns 179.11 ns 1.22 0.08 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).MultiplyOperatorBenchmark.html>) 136.25 ns 151.71 ns 1.11 0.08 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).GreaterThanOrEqualAnyBenchmark.html>) 114.83 ns 283.62 ns 2.47 0.10 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).AndNotBenchmark.html>) 171.29 ns 272.29 ns 1.59 0.08 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).AddBenchmark.html>) 137.13 ns 169.73 ns 1.24 0.09 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).LessThanOrEqualAnyBenchmark.html>) 52.71 ns 226.59 ns 4.30 0.12 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).GreaterThanOrEqualAllBenchmark.html>) 53.55 ns 314.02 ns 5.86 0.07 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).OnesComplementBenchmark.html>) 67.08 ns 159.14 ns 2.37 0.07 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).LessThanAnyBenchmark.html>) 51.35 ns 226.02 ns 4.40 0.09 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).UnaryNegateOperatorBenchmark.html>) 137.93 ns 168.75 ns 1.22 0.10 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).SubtractBenchmark.html>) 136.60 ns 171.24 ns 1.25 0.08 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).NegateBenchmark.html>) 143.57 ns 189.10 ns 1.32 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).LessThanOrEqualAllBenchmark.html>) 117.66 ns 361.90 ns 3.08 0.13 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).LessThanAllBenchmark.html>) 108.67 ns 356.20 ns 3.28 0.12 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).GreaterThanAllBenchmark.html>) 52.88 ns 306.24 ns 5.79 0.12 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).EqualsAnyBenchmark.html>) 112.98 ns 259.57 ns 2.30 0.14 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).AddOperatorBenchmark.html>) 137.56 ns 154.77 ns 1.13 0.09 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).EqualsBenchmark.html>) 81.00 ns 88.45 ns 1.09 0.16 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).DivisionOperatorBenchmark.html>) 144.65 ns 162.21 ns 1.12 0.13 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).OneBenchmark.html>) 54.07 ns 95.32 ns 1.76 0.10 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).SubtractionOperatorBenchmark.html>) 136.21 ns 152.34 ns 1.12 0.06 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).GreaterThanAnyBenchmark.html>) 109.34 ns 279.45 ns 2.56 0.11 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt32).OnesComplementOperatorBenchmark.html>) 67.68 ns 161.81 ns 2.39 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt32>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 179.10740141626022 > 153.359567456022. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.216660295439915 (T) = (0 -179.80069671910888) / Math.Sqrt((4.248252241294571 / (44)) + (11.5941722841697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2234477715851106 = (146.96229859175548 - 179.80069671910888) / 146.96229859175548 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.71131347582462 > 143.01722749670378. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.003578280687933 (T) = (0 -153.65750617661655) / Math.Sqrt((6.713392151199911 / (44)) + (10.783470912140961 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12281161825230015 = (136.8506556921725 - 153.65750617661655) / 136.8506556921725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 283.6191464725277 > 121.72548485403605. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -191.19460797420754 (T) = (0 -284.19006002117953) / Math.Sqrt((16.87368208205776 / (44)) + (5.743425291716494 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.4328953592509546 = (116.81146044385427 - 284.19006002117953) / 116.81146044385427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.292802010028 > 179.2496569396661. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -155.0772462775444 (T) = (0 -273.39812365277515) / Math.Sqrt((6.238323406749256 / (44)) + (4.519316564622393 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6065843498467258 = (170.1735260142789 - 273.39812365277515) / 170.1735260142789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.72632074796482 > 143.6919170350379. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -86.13828141048239 (T) = (0 -169.7736153826015) / Math.Sqrt((2.7853856340475422 / (44)) + (1.1991570306404147 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2376705546464172 = (137.17189501296903 - 169.7736153826015) / 137.17189501296903 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 226.59365392521235 > 55.448567543295766. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -234.93468674506278 (T) = (0 -226.92532760268895) / Math.Sqrt((1.77418980947464 / (44)) + (7.557710017798118 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.2321797914556436 = (53.619018752660025 - 226.92532760268895) / 53.619018752660025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.02420496355654 > 56.298097388013566. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -548.3643887737626 (T) = (0 -313.6568059755372) / Math.Sqrt((0.8497211972014314 / (44)) + (3.0843252097961864 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.8536447261235685 = (53.58316410556208 - 313.6568059755372) / 53.58316410556208 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 159.1428906202502 > 70.6476582503671. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.375786978086424 (T) = (0 -160.86278267388795) / Math.Sqrt((27.671884814877533 / (44)) + (31.049548802261324 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3570609272885363 = (68.24718903593966 - 160.86278267388795) / 68.24718903593966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 226.0165422854005 > 53.81543691828992. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -114.41150048330209 (T) = (0 -222.69859804842767) / Math.Sqrt((5.135637855315282 / (44)) + (29.507627038777855 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -3.2777134616307713 = (52.060195252893216 - 222.69859804842767) / 52.060195252893216 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 168.74648202756393 > 143.51917048091963. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.90736001423888 (T) = (0 -168.61409093348894) / Math.Sqrt((10.773211390294136 / (44)) + (1.413547715484338 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2392736326433262 = (136.0588061361728 - 168.61409093348894) / 136.0588061361728 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 171.2398123530384 > 144.47051060589135. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.95596342047286 (T) = (0 -170.5090100569022) / Math.Sqrt((10.270082559757766 / (44)) + (3.416060802286729 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23608201697318046 = (137.94312004832102 - 170.5090100569022) / 137.94312004832102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 189.09943902111144 > 141.60859965551765. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.56956048974644 (T) = (0 -171.9099133485132) / Math.Sqrt((16.61828952093992 / (44)) + (48.23239777814865 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2574784134699831 = (136.71003136676651 - 171.9099133485132) / 136.71003136676651 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.89778325123154 > 121.51925870841166. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -111.82077564499347 (T) = (0 -368.9610311725855) / Math.Sqrt((9.132754267900168 / (44)) + (73.34093159333463 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.166789562726814 = (116.50948819437366 - 368.9610311725855) / 116.50948819437366 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 356.1958745457678 > 114.96596662818028. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -214.58578389678544 (T) = (0 -358.5450682069296) / Math.Sqrt((4.93491907183325 / (44)) + (18.46733782233242 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.264332649186996 = (109.8371724757364 - 358.5450682069296) / 109.8371724757364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 306.2402729302095 > 55.20585630479445. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -132.94390544906324 (T) = (0 -311.8921507302935) / Math.Sqrt((3.8752773412099577 / (43)) + (55.83156629155887 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -4.961222488367943 = (52.32016609661603 - 311.8921507302935) / 52.32016609661603 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 259.5746972910727 > 114.7432795325476. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -77.31963750535867 (T) = (0 -261.44878235765356) / Math.Sqrt((124.28630489523374 / (43)) + (12.216434149512004 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -1.345652392610859 = (111.4610089633292 - 261.44878235765356) / 111.4610089633292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.7711562072944 > 143.88451054495408. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.9273272850588 (T) = (0 -153.12321497220915) / Math.Sqrt((4.9464716112892155 / (44)) + (1.200337690297364 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11301363418831566 = (137.57532726351272 - 153.12321497220915) / 137.57532726351272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.45199605477073 > 86.16935923516589. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.58636370668067 (T) = (0 -88.6067666887679) / Math.Sqrt((3.1527777953406853 / (44)) + (2.5892064999622333 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0755056034217916 = (82.38615066891299 - 88.6067666887679) / 82.38615066891299 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 162.206496809931 > 152.85830778746345. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.540661999503227 (T) = (0 -162.960689748436) / Math.Sqrt((7.25188192496677 / (44)) + (14.442765145307236 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11268012852527823 = (146.45780541117466 - 162.960689748436) / 146.45780541117466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 95.31639676410684 > 56.89068325603382. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.30216212026296 (T) = (0 -93.56350613946414) / Math.Sqrt((13.910248190082838 / (44)) + (4.606619917248826 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6879337686737441 = (55.430792295232976 - 93.56350613946414) / 55.430792295232976 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.3388009990242 > 142.9505395699193. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.18309072077681 (T) = (0 -152.60906933680195) / Math.Sqrt((4.288225608767016 / (44)) + (2.4202438290260995 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11601041152278038 = (136.7452021604073 - 152.60906933680195) / 136.7452021604073 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.446480466049 > 114.82847182266457. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -159.5564210901143 (T) = (0 -279.88889297476857) / Math.Sqrt((23.18640090617761 / (44)) + (8.942732206981843 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5262263937818086 = (110.7932739772264 - 279.88889297476857) / 110.7932739772264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt32>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 161.8096632604325 > 69.69818852852185. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.122381393589144 (T) = (0 -161.34693308590437) / Math.Sqrt((2.7542250766802003 / (44)) + (58.062085963974816 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3765924407245043 = (67.89003041544547 - 161.34693308590437) / 67.89003041544547 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromReader(Mode%3a%20Reflection).html>) 6.59 μs 7.94 μs 1.21 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromString(Mode%3a%20SourceGen).html>) 7.81 μs 9.18 μs 1.18 0.04 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 6.77 μs 8.06 μs 1.19 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 4.21 μs 5.22 μs 1.24 0.09 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 4.39 μs 5.16 μs 1.17 0.08 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Nullable(DateTimeOffset)).DeserializeFromString(Mode%3a%20Reflection).html>) 7.81 μs 9.18 μs 1.18 0.01 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) graph_6.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.938744732067513 > 7.100442241669304. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.32423749155124 (T) = (0 -8032.190042568297) / Math.Sqrt((13122.855985410151 / (44)) + (9691.229862096316 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19321214977021003 = (6731.569104551227 - 8032.190042568297) / 6731.569104551227 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.180924972911741 > 8.219394322508343. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.50796283710214 (T) = (0 -9173.957043537308) / Math.Sqrt((10082.94393828986 / (43)) + (7941.70238917005 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.17115915845232163 = (7833.228282704655 - 9173.957043537308) / 7833.228282704655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.06168128545642 > 7.089753828106155. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.664138604232 (T) = (0 -7999.873803865631) / Math.Sqrt((12545.592123774184 / (44)) + (10394.738334122992 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1835204317930952 = (6759.387999534072 - 7999.873803865631) / 6759.387999534072 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.222595521794027 > 4.468565209085506. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.712781630285907 (T) = (0 -5153.957536153519) / Math.Sqrt((6710.800957588984 / (44)) + (14664.51979658469 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21100503600426437 = (4255.9340241549335 - 5153.957536153519) / 4255.9340241549335 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.160043159307215 > 4.518977950571349. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.883220789074066 (T) = (0 -5171.881279165389) / Math.Sqrt((8915.510546208538 / (44)) + (7646.543648878213 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21191912553462222 = (4267.513541288394 - 5171.881279165389) / 4267.513541288394 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.180647211822102 > 8.196525136825391. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.186750679754944 (T) = (0 -9225.383225655134) / Math.Sqrt((7404.022957890037 / (43)) + (5759.1095322623105 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.17826695902476625 = (7829.620575366761 - 9225.383225655134) / 7829.620575366761 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetAddressBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address%3a%20143.24.20.36).html>) 106.18 ns 124.18 ns 1.17 0.11 False
[Ctor_Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address%3a%20%5b143%2c%2024%2c%2020%2c%2036%5d).html>) 106.69 ns 140.33 ns 1.32 0.16 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address%3a%201020%3a3040%3a5060%3a7080%3a9010%3a1112%3a1314%3a1516).html>) 3.39 μs 3.85 μs 1.14 0.03 False
[Ctor_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address%3a%20%5b143%2c%2024%2c%2020%2c%2036%5d).html>) 120.02 ns 167.33 ns 1.39 0.15 False
[TryWriteBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address%3a%20143.24.20.36).html>) 89.02 ns 119.58 ns 1.34 0.17 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address%3a%20143.24.20.36).html>) 570.27 ns 623.38 ns 1.09 0.08 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 143.24.20.36) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.18329291636952 > 110.96170447397063. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.799362226776058 (T) = (0 -125.62372892526518) / Math.Sqrt((3.7729029324445826 / (44)) + (19.761523228729995 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1882247943781483 = (105.72387440459846 - 125.62372892526518) / 105.72387440459846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [143, 24, 20, 36]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 140.32611693346558 > 111.46406704197015. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.57116585743468 (T) = (0 -142.94294252208283) / Math.Sqrt((4.238865818008326 / (44)) + (32.99815680824246 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.352222918032974 = (105.70959907262655 - 142.94294252208283) / 105.70959907262655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 1020:3040:5060:7080:9010:1112:1314:1516) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.8523914595244015 > 3.6043815532254344. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.878639873624817 (T) = (0 -3858.3231552104317) / Math.Sqrt((3835.6492667714583 / (44)) + (1654.558122595445 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10852041311452146 = (3480.6063195264114 - 3858.3231552104317) / 3480.6063195264114 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [143, 24, 20, 36]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.32641472228076 > 132.6328985491252. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.727202891784124 (T) = (0 -168.39987364027903) / Math.Sqrt((9.149535538052469 / (44)) + (94.17641100023013 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.312084725822994 = (128.34527399490275 - 168.39987364027903) / 128.34527399490275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 143.24.20.36) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.58154913412653 > 94.29896521453038. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.841146112046175 (T) = (0 -120.8359264442145) / Math.Sqrt((1.535757085603573 / (44)) + (32.46047296463056 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34365163346330063 = (89.93099359597876 - 120.8359264442145) / 89.93099359597876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 143.24.20.36) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 623.3816453808665 > 577.3689307321456. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.21195907159949 (T) = (0 -626.2790633812072) / Math.Sqrt((168.5011629053152 / (44)) + (25.612783743956026 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13285166626196113 = (552.8341282735829 - 626.2790633812072) / 552.8341282735829 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Utf8JsonReaderCommentsTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 1.66 μs 1.84 μs 1.11 0.05 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 1.71 μs 1.87 μs 1.09 0.08 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortMultiLine).html>) 2.15 μs 2.38 μs 1.10 0.06 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortSingleLine).html>) 1.92 μs 2.06 μs 1.07 0.08 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortSingleLine).html>) 2.06 μs 2.21 μs 1.07 0.06 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortSingleLine).html>) 1.58 μs 1.73 μs 1.10 0.05 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Allow%2c%20SegmentSize%3a%200%2c%20TestCase%3a%20ShortMultiLine).html>) 1.82 μs 2.01 μs 1.10 0.04 False
[Utf8JsonReaderCommentParsing - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling%3a%20Skip%2c%20SegmentSize%3a%20100%2c%20TestCase%3a%20ShortMultiLine).html>) 2.12 μs 2.35 μs 1.11 0.05 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Utf8JsonReaderCommentsTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8447284265969668 > 1.799244223476877. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.710495866301482 (T) = (0 -1849.8329745880828) / Math.Sqrt((814.960355887486 / (44)) + (960.4678350057211 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08941640424687915 = (1698.0035984191782 - 1849.8329745880828) / 1698.0035984191782 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8691594814163226 > 1.799000978368545. IsChangePoint: Marked as a change because one of 9/24/2022 7:43:48 AM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.32905738480026 (T) = (0 -1897.8007414568153) / Math.Sqrt((747.1853226686345 / (44)) + (1671.69613850529 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11170960081229545 = (1707.1011530980256 - 1897.8007414568153) / 1707.1011530980256 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3783656052022746 > 2.2971437862527746. IsChangePoint: Marked as a change because one of 9/27/2022 3:32:36 AM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.598564670131523 (T) = (0 -2392.6173594069164) / Math.Sqrt((1956.9532076894795 / (44)) + (2226.9015020777847 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09259862589955477 = (2189.8410840824868 - 2392.6173594069164) / 2189.8410840824868 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0557962098620615 > 2.0234993057931905. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.981871647431912 (T) = (0 -2102.6691989470633) / Math.Sqrt((1071.1972701442198 / (44)) + (2940.7830307035842 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09241553980699173 = (1924.7888027284594 - 2102.6691989470633) / 1924.7888027284594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 100, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2060154019763325 > 2.152320909331535. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 8:25:19 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.542722625726912 (T) = (0 -2256.0322535249657) / Math.Sqrt((792.5775739931006 / (42)) + (8289.163344287479 / (17))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (17) - 2, .025) and -0.10520761802714068 = (2041.274613680381 - 2256.0322535249657) / 2041.274613680381 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 0, TestCase: ShortSingleLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7318901235590909 > 1.6638081966009401. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.565542471500653 (T) = (0 -1715.6724873428625) / Math.Sqrt((795.6548244315192 / (44)) + (747.1426349991426 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08643424791164221 = (1579.1774703722292 - 1715.6724873428625) / 1579.1774703722292 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Allow, SegmentSize: 0, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.012164575336113 > 1.9132670758020496. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.626390301428266 (T) = (0 -2025.4216361945537) / Math.Sqrt((1124.0951949908829 / (44)) + (3756.0334447963764 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10540627296321857 = (1832.287083702797 - 2025.4216361945537) / 1832.287083702797 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Utf8JsonReaderCommentsTests.Utf8JsonReaderCommentParsing(CommentHandling: Skip, SegmentSize: 100, TestCase: ShortMultiLine) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3479508114326495 > 2.1541175136523347. IsChangePoint: Marked as a change because one of 9/28/2022 11:19:50 AM, 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.95791479652853 (T) = (0 -2291.8662345272433) / Math.Sqrt((1149.4014541706401 / (44)) + (1162.5824103404123 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10807081778940476 = (2068.339132962191 - 2291.8662345272433) / 2068.339132962191 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 311.29 μs 364.37 μs 1.17 0.04 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20Reflection).html>) 332.19 μs 389.51 μs 1.17 0.05 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 307.46 μs 358.57 μs 1.17 0.05 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToWriter(Mode%3a%20SourceGen).html>) 315.42 μs 365.58 μs 1.16 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToWriter(Mode%3a%20Reflection).html>) 310.27 μs 358.10 μs 1.15 0.02 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 362.22 μs 405.07 μs 1.12 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeToString(Mode%3a%20SourceGen).html>) 336.94 μs 398.94 μs 1.18 0.04 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(ArrayList).SerializeObjectProperty(Mode%3a%20Reflection).html>) 355.14 μs 406.58 μs 1.14 0.04 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ArrayList&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.3667428861788 > 328.1212017493873. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.64995057794935 (T) = (0 -363199.88541427907) / Math.Sqrt((21376595.95938648 / (43)) + (10183422.089732125 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16994592547647097 = (310441.6003383769 - 363199.88541427907) / 310441.6003383769 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 389.51036346153853 > 356.5124935913322. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.325582628673324 (T) = (0 -391596.0871334187) / Math.Sqrt((29754799.55159939 / (44)) + (28619722.526688285 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1530045573582622 = (339630.9968025059 - 391596.0871334187) / 339630.9968025059 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 358.5687919034092 > 328.6959877283654. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.78895031295418 (T) = (0 -363043.42956892285) / Math.Sqrt((15940299.513881825 / (43)) + (13315836.575711032 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.16062719389387095 = (312799.3480412281 - 363043.42956892285) / 312799.3480412281 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 365.5762520125224 > 326.5451227419117. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.60196428344641 (T) = (0 -360620.03598865867) / Math.Sqrt((17515681.723460563 / (44)) + (8815048.855927365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16559545487053404 = (309386.9613867993 - 360620.03598865867) / 309386.9613867993 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 358.0992424242425 > 324.7766076746323. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.75940213326372 (T) = (0 -360923.8946874146) / Math.Sqrt((19592853.29580883 / (44)) + (14713440.958973432 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1647993638408996 = (309859.2821147122 - 360923.8946874146) / 309859.2821147122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 405.0719854469855 > 370.4180374097223. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.95756780060445 (T) = (0 -404184.11189331696) / Math.Sqrt((32676042.794655666 / (44)) + (21337451.328964088 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14582141371226845 = (352746.16712200246 - 404184.11189331696) / 352746.16712200246 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 398.940156882591 > 355.9412517482517. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.42167651351447 (T) = (0 -396314.83788012265) / Math.Sqrt((24594472.671763383 / (43)) + (21504467.607875865 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.16381559173753435 = (340530.61386507034 - 396314.83788012265) / 340530.61386507034 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<ArrayList>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 406.57692266483514 > 366.97989522892453. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.26120071921722 (T) = (0 -403996.626852834) / Math.Sqrt((25947076.76234 / (44)) + (33291461.85525325 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15452247109087414 = (349925.3041572327 - 403996.626852834) / 349925.3041572327 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Node.Tests.Perf_ParseThenWrite

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 1.08 ms 1.58 ms 1.46 0.03 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 1.98 ms 2.76 ms 1.40 0.05 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 1.47 ms 2.09 ms 1.42 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json400B).html>) 212.80 μs 292.11 μs 1.37 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 1.21 ms 1.76 ms 1.45 0.02 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 297.78 μs 447.04 μs 1.50 0.03 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 34.44 μs 46.38 μs 1.35 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 1.58 ms 2.21 ms 1.40 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 443.20 μs 619.97 μs 1.40 0.02 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json400KB).html>) 148.51 ms 207.04 ms 1.39 0.02 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 2.11 ms 3.05 ms 1.45 0.02 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 257.50 μs 425.40 μs 1.65 0.04 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 135.90 ms 193.64 ms 1.42 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 469.55 μs 652.03 μs 1.39 0.01 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20Json400B).html>) 228.39 μs 308.28 μs 1.35 0.02 False
[ParseThenWrite - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 37.02 μs 49.40 μs 1.33 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Node.Tests.Perf_ParseThenWrite*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.579328057553957 > 1.1437696688393466. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -120.98553365329622 (T) = (0 -1581947.2367573446) / Math.Sqrt((437210284.1668993 / (44)) + (96069987.90928484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.44751461094420175 = (1092871.3429189178 - 1581947.2367573446) / 1092871.3429189178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.7634015011235955 > 2.004927703819804. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.70440317592708 (T) = (0 -2776058.995841776) / Math.Sqrt((2188453892.894832 / (44)) + (429137669.4248573 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.45924831940099287 = (1902389.7159472632 - 2776058.995841776) / 1902389.7159472632 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.089061015819209 > 1.5427851985207104. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -90.79709598175988 (T) = (0 -2091861.1326050672) / Math.Sqrt((1089619478.5654736 / (43)) + (288727845.9169576 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.41697143214398075 = (1476290.26609233 - 2091861.1326050672) / 1476290.26609233 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 292.1063746543779 > 224.77776340378213. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -105.97687725354338 (T) = (0 -292187.8344345187) / Math.Sqrt((9695750.971459948 / (44)) + (4719771.29265375 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3610912169038191 = (214671.75072893445 - 292187.8344345187) / 214671.75072893445 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7582946377622377 > 1.2900044466782807. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -133.95023153735528 (T) = (0 -1749454.268657415) / Math.Sqrt((371384122.4443759 / (44)) + (101756804.17810442 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42602793333959965 = (1226802.2440208353 - 1749454.268657415) / 1226802.2440208353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 447.04211711711724 > 301.2399144270833. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -139.91743490174886 (T) = (0 -452553.9744690614) / Math.Sqrt((43321394.250268005 / (44)) + (7027642.758363716 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5941070184985848 = (283891.83989372366 - 452553.9744690614) / 283891.83989372366 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 46.38091911764705 > 36.16430797848896. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -86.36719759321602 (T) = (0 -46721.12757812351) / Math.Sqrt((131825.81003061618 / (44)) + (260310.845766775 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3581517274175879 = (34400.521410785106 - 46721.12757812351) / 34400.521410785106 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.2106625017857144 > 1.6554717140293798. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.42096946316035 (T) = (0 -2222524.8408777374) / Math.Sqrt((847441130.3773062 / (44)) + (335463089.3993301 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3999770153505508 = (1587543.8071540215 - 2222524.8408777374) / 1587543.8071540215 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 619.9682079480941 > 436.0338785356405. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -100.12464046092127 (T) = (0 -625018.3409930207) / Math.Sqrt((51618368.4030394 / (44)) + (48417621.16393142 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5061714044894629 = (414971.5889772048 - 625018.3409930207) / 414971.5889772048 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 207.04421442857142 > 156.30198738750002. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.6368842492976 (T) = (0 -207551838.1361905) / Math.Sqrt((9284635352645.793 / (44)) + (2011291967037.4824 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38188435532812 = (150194795.4877227 - 207551838.1361905) / 150194795.4877227 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0529399168724276 > 2.209411061283887. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -143.99346402484036 (T) = (0 -3024839.5077953194) / Math.Sqrt((1149902086.6790018 / (44)) + (213489919.1679415 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4335875778737036 = (2109978.8771061758 - 3024839.5077953194) / 2109978.8771061758 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 425.3972671038252 > 270.3361017033812. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -109.89021724099574 (T) = (0 -422917.0781189491) / Math.Sqrt((39704486.90685399 / (44)) + (19012892.4486341 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6201088969801322 = (261042.37740269347 - 422917.0781189491) / 261042.37740269347 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 193.6387336 > 144.69696362500002. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.44871163276278 (T) = (0 -195291354.3816361) / Math.Sqrt((7422150657344.008 / (43)) + (6218596631391.595 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.4048802161785963 = (139009256.54206064 - 195291354.3816361) / 139009256.54206064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 652.032646563574 > 474.37376971942456. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -150.68815387509164 (T) = (0 -657126.0952808826) / Math.Sqrt((38370426.258078285 / (44)) + (14564294.93521601 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4520335597540052 = (452555.7215028893 - 657126.0952808826) / 452555.7215028893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 308.27658993743484 > 240.10589816087497. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -123.17683499181466 (T) = (0 -309211.0850888705) / Math.Sqrt((9532981.848829838 / (44)) + (3082712.7580829286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3492230429031729 = (229177.1451097734 - 309211.0850888705) / 229177.1451097734 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Node.Tests.Perf_ParseThenWrite.ParseThenWrite(IsDataIndented: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.39620861801242 > 38.71582034959213. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -117.3724584288723 (T) = (0 -49148.27540298146) / Math.Sqrt((175306.5851715751 / (44)) + (117021.35301836526 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35000073101470064 = (36406.11021450348 - 49148.27540298146) / 36406.11021450348 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 16.86 ms 19.59 ms 1.16 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20Reflection).html>) 17.95 ms 20.49 ms 1.14 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 16.15 ms 18.70 ms 1.16 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 17.99 ms 20.28 ms 1.13 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 16.40 ms 18.98 ms 1.16 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 17.95 ms 20.50 ms 1.14 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 16.55 ms 18.80 ms 1.14 0.01 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.586032474025973 > 17.766106504423078. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.272653750662 (T) = (0 -19190741.877858344) / Math.Sqrt((51471246558.92702 / (44)) + (19130762034.657787 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1454240999165505 = (16754267.593336372 - 19190741.877858344) / 16754267.593336372 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.48526061818181 > 18.9377761675. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.38433809547321 (T) = (0 -20551838.491250973) / Math.Sqrt((43316355028.90579 / (44)) + (21304983485.33006 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14597456080112198 = (17933939.54302415 - 20551838.491250973) / 17933939.54302415 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.70010557777778 > 17.13990117. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.836165258526876 (T) = (0 -18623121.189463414) / Math.Sqrt((63663418348.27929 / (44)) + (6975505276.853781 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14924640982393306 = (16204637.256440518 - 18623121.189463414) / 16204637.256440518 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.282284830303027 > 18.804991819230768. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.19748978376213 (T) = (0 -20211900.12463564) / Math.Sqrt((64060256045.25784 / (44)) + (6247373851.555329 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14512763316630245 = (17650347.034897152 - 20211900.12463564) / 17650347.034897152 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.98142222222222 > 17.361141348461537. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.65011283406591 (T) = (0 -18897218.920799755) / Math.Sqrt((51792529623.537506 / (44)) + (13357431976.597065 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14506576676457134 = (16503173.415265566 - 18897218.920799755) / 16503173.415265566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.504077902597405 > 18.933584661723376. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.15059055856766 (T) = (0 -20517939.54491065) / Math.Sqrt((44901650276.521 / (44)) + (13993502562.156263 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14038960557791627 = (17992043.635396656 - 20517939.54491065) / 17992043.635396656 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.7970889 > 17.418401171499998. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.22626048606225 (T) = (0 -18931909.26899674) / Math.Sqrt((52067593601.9823 / (44)) + (24670848142.131317 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14922637690322685 = (16473611.857057946 - 18931909.26899674) / 16473611.857057946 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).AbsBenchmark.html>) 200.66 ns 290.57 ns 1.45 0.06 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanBenchmark.html>) 261.99 ns 313.26 ns 1.20 0.09 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).MaxBenchmark.html>) 295.44 ns 347.29 ns 1.18 0.10 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).UnaryNegateOperatorBenchmark.html>) 264.49 ns 330.83 ns 1.25 0.10 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).EqualsBenchmark.html>) 84.16 ns 92.99 ns 1.10 0.14 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanAnyBenchmark.html>) 51.04 ns 365.70 ns 7.16 0.10 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).NegateBenchmark.html>) 244.40 ns 333.43 ns 1.36 0.09 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanAnyBenchmark.html>) 194.06 ns 427.01 ns 2.20 0.13 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).OnesComplementBenchmark.html>) 67.22 ns 228.76 ns 3.40 0.10 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).DivisionOperatorBenchmark.html>) 262.71 ns 330.39 ns 1.26 0.11 False
[GreaterThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanOrEqualBenchmark.html>) 279.60 ns 330.15 ns 1.18 0.14 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).OneBenchmark.html>) 88.80 ns 167.81 ns 1.89 0.07 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).MinBenchmark.html>) 303.15 ns 355.28 ns 1.17 0.06 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).OnesComplementOperatorBenchmark.html>) 67.26 ns 235.14 ns 3.50 0.08 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanAllBenchmark.html>) 50.73 ns 469.61 ns 9.26 0.09 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanOrEqualAllBenchmark.html>) 52.84 ns 471.53 ns 8.92 0.07 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).SubtractBenchmark.html>) 245.18 ns 330.73 ns 1.35 0.07 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).SquareRootBenchmark.html>) 223.97 ns 275.22 ns 1.23 0.07 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanOrEqualBenchmark.html>) 270.36 ns 323.39 ns 1.20 0.10 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanOrEqualAnyBenchmark.html>) 207.06 ns 440.77 ns 2.13 0.10 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).ConditionalSelectBenchmark.html>) 349.27 ns 504.21 ns 1.44 0.11 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).AddBenchmark.html>) 244.78 ns 333.51 ns 1.36 0.10 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanOrEqualAnyBenchmark.html>) 72.36 ns 377.57 ns 5.22 0.13 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).AndNotBenchmark.html>) 169.94 ns 339.65 ns 2.00 0.12 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).DivideBenchmark.html>) 262.50 ns 361.86 ns 1.38 0.09 False
[GreaterThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).GreaterThanBenchmark.html>) 292.97 ns 313.91 ns 1.07 0.10 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanOrEqualAllBenchmark.html>) 206.45 ns 518.90 ns 2.51 0.14 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).SubtractionOperatorBenchmark.html>) 245.46 ns 314.19 ns 1.28 0.10 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).LessThanAllBenchmark.html>) 193.62 ns 503.49 ns 2.60 0.07 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).MultiplyOperatorBenchmark.html>) 244.19 ns 312.56 ns 1.28 0.10 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).EqualsAnyBenchmark.html>) 195.52 ns 428.85 ns 2.19 0.09 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).MultiplyBenchmark.html>) 246.59 ns 332.34 ns 1.35 0.11 False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).EqualsStaticBenchmark.html>) 271.40 ns 317.33 ns 1.17 0.11 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Int16).AddOperatorBenchmark.html>) 244.35 ns 315.02 ns 1.29 0.07 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Int16>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 290.5696710021795 > 210.42134628817163. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -72.77645107290962 (T) = (0 -287.69232006322176) / Math.Sqrt((27.049458019557367 / (44)) + (11.923227493000336 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42927112656864796 = (201.28603643866018 - 287.69232006322176) / 201.28603643866018 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 313.25939798264284 > 274.8446613418314. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.16064350149937 (T) = (0 -314.6643061441601) / Math.Sqrt((595.899363852905 / (44)) + (41.711928745207864 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1850345779063149 = (265.531750727561 - 314.6643061441601) / 265.531750727561 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.29378317815923 > 310.9535579678219. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.601335538045205 (T) = (0 -347.93804762602315) / Math.Sqrt((95.16378405977085 / (43)) + (1.8655314755787997 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15535608506926252 = (301.15221802390437 - 347.93804762602315) / 301.15221802390437 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 330.82979122643576 > 257.1322733727401. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.777208977424856 (T) = (0 -331.7971882743212) / Math.Sqrt((337.7925748998593 / (44)) + (8.398873580593122 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.33141758898848833 = (249.20595237621572 - 331.7971882743212) / 249.20595237621572 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 92.98525683132276 > 88.86082379120766. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.360269815698764 (T) = (0 -94.94883818753947) / Math.Sqrt((2.997415832732158 / (44)) + (2.679429519243846 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12580905900960154 = (84.33831423515814 - 94.94883818753947) / 84.33831423515814 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 365.70027979373566 > 53.504289880501254. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -332.78087439168013 (T) = (0 -367.92450569288116) / Math.Sqrt((1.41545789500347 / (44)) + (13.049203952935912 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -6.095958934727932 = (51.84986399685074 - 367.92450569288116) / 51.84986399685074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 333.4306431353616 > 257.3573966644005. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.361605212209014 (T) = (0 -331.3997436521158) / Math.Sqrt((35.37965318751514 / (44)) + (21.791836998630014 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3431767221579575 = (246.72832560683938 - 331.3997436521158) / 246.72832560683938 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 427.0113445161914 > 203.97256404040672. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -149.05040322846665 (T) = (0 -427.62032023436296) / Math.Sqrt((101.26646830294942 / (44)) + (1.3883180820435863 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1706824110546823 = (196.9981044009992 - 427.62032023436296) / 196.9981044009992 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.75614255479908 > 70.35106849614414. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -85.20449493962224 (T) = (0 -233.8823209910785) / Math.Sqrt((2.091773224754567 / (44)) + (56.44245672237334 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.46175947760573 = (67.56169008969954 - 233.8823209910785) / 67.56169008969954 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 330.39380740871593 > 278.2453727508059. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.720756286473463 (T) = (0 -332.5517326084667) / Math.Sqrt((81.84980498041494 / (43)) + (31.850083464789666 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.24220788204401253 = (267.7102097124546 - 332.5517326084667) / 267.7102097124546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 330.1512292813491 > 293.56682456780266. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.5639590022166 (T) = (0 -331.5969999489891) / Math.Sqrt((62.66105754134159 / (44)) + (24.78247192702566 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17067976069650342 = (283.25167230336615 - 331.5969999489891) / 283.25167230336615 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 167.81229332508283 > 92.24273409806936. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -107.34536770163831 (T) = (0 -163.6030963657405) / Math.Sqrt((3.054410862424744 / (44)) + (6.280478386257215 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.846413987803793 = (88.60585840791713 - 163.6030963657405) / 88.60585840791713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 355.2810119020678 > 317.89538493326535. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.8856632770963 (T) = (0 -356.8151822433913) / Math.Sqrt((6.821313564410848 / (43)) + (67.09754860038932 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.17660917635422746 = (303.25718124092657 - 356.8151822433913) / 303.25718124092657 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.13511685871308 > 70.40589861194417. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -194.3859869741912 (T) = (0 -231.2702098816679) / Math.Sqrt((0.9460529828104406 / (44)) + (10.34288019777169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.4333820640357215 = (67.3592992472922 - 231.2702098816679) / 67.3592992472922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 469.6129805374752 > 53.91000794460748. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -116.52921974918402 (T) = (0 -467.4474061374744) / Math.Sqrt((1.914953355416035 / (44)) + (189.85390851072134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -7.961077773079758 = (52.164194751411166 - 467.4474061374744) / 52.164194751411166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 471.5254965352501 > 55.490896651377575. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -246.47333741604564 (T) = (0 -481.90590908257195) / Math.Sqrt((0.852898416395473 / (44)) + (45.03218475560175 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -8.01222789498622 = (53.47245039716218 - 481.90590908257195) / 53.47245039716218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 330.726690803954 > 259.4119792200845. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.57744236983837 (T) = (0 -333.2427732364803) / Math.Sqrt((324.209056483952 / (44)) + (18.4196822112638 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3358834937053589 = (249.4549672982036 - 333.2427732364803) / 249.4549672982036 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 275.2208400520864 > 235.71113272168293. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -124.83376395636546 (T) = (0 -273.8564879333172) / Math.Sqrt((3.768226885629218 / (44)) + (0.9802086464092892 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21525229766001155 = (225.3494920031276 - 273.8564879333172) / 225.3494920031276 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 323.3881476366552 > 285.2762252796735. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.84820500458227 (T) = (0 -324.3691808395777) / Math.Sqrt((50.093858836526515 / (44)) + (15.98886744400327 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1769628535523499 = (275.59848627384923 - 324.3691808395777) / 275.59848627384923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 440.7719738687075 > 218.0040728549014. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -171.42006035529448 (T) = (0 -443.0085779775167) / Math.Sqrt((12.932067148077978 / (44)) + (23.835363827012 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1320440746341738 = (207.78584422722605 - 443.0085779775167) / 207.78584422722605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 504.20908062206894 > 367.44794078816227. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -88.38365813659814 (T) = (0 -497.70113188510317) / Math.Sqrt((26.911786238409245 / (44)) + (31.29962993848758 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4118443985946358 = (352.5184024390506 - 497.70113188510317) / 352.5184024390506 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 333.5056864395267 > 257.3693317535926. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.478659239146516 (T) = (0 -334.3751503957763) / Math.Sqrt((9.28019105458276 / (44)) + (32.07477388929735 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3558250875561736 = (246.62115597704096 - 334.3751503957763) / 246.62115597704096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 377.5725945142868 > 56.105958012337176. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -149.6774458768571 (T) = (0 -381.9720648190843) / Math.Sqrt((9.231626386641485 / (43)) + (64.28402638703584 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -6.096358420001234 = (53.826489899733375 - 381.9720648190843) / 53.826489899733375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 339.65119025446415 > 178.2486297502153. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -189.823452801698 (T) = (0 -341.3317641529539) / Math.Sqrt((9.250345220478573 / (44)) + (9.026780673945504 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0045592488565305 = (170.2777128426916 - 341.3317641529539) / 170.2777128426916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.86091992748754 > 276.73678508251106. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.344777382443986 (T) = (0 -350.78876940236836) / Math.Sqrt((37.29445440613405 / (44)) + (20.405200549907022 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.32251993624304154 = (265.24270809775027 - 350.78876940236836) / 265.24270809775027 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.GreaterThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 313.9097509059949 > 283.12236186497034. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.283223136040835 (T) = (0 -318.32414511896934) / Math.Sqrt((36.55395040318886 / (44)) + (33.39979043217271 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18393532967602555 = (268.8695380059958 - 318.32414511896934) / 268.8695380059958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 518.9032880171795 > 216.30720891380224. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.75159768805167 (T) = (0 -533.7304452165905) / Math.Sqrt((38.437997838180976 / (44)) + (817.2194189985339 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5633648498599844 = (208.21477880752863 - 533.7304452165905) / 208.21477880752863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.18629925039596 > 257.69961975471756. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.02544367833081 (T) = (0 -315.4918480572814) / Math.Sqrt((27.62978997135805 / (44)) + (8.255436049521288 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2713080794899245 = (248.16317393645713 - 315.4918480572814) / 248.16317393645713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 503.48898316255196 > 203.38172105025689. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -168.89301531338822 (T) = (0 -512.0622729079773) / Math.Sqrt((10.069598470459637 / (44)) + (49.5766353350134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.6318582994673378 = (194.56301010263877 - 512.0622729079773) / 194.56301010263877 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 312.56220713009884 > 258.0019795190827. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.60447589874262 (T) = (0 -316.1669585706302) / Math.Sqrt((12.868325365145973 / (44)) + (21.12938973528941 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2771472468098315 = (247.55717037356442 - 316.1669585706302) / 247.55717037356442 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 428.84518505893624 > 204.33611237959377. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -80.24259814742213 (T) = (0 -433.97237882298015) / Math.Sqrt((80.19029782239411 / (44)) + (103.77071679128296 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.205819844059291 = (196.73972015065218 - 433.97237882298015) / 196.73972015065218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 332.34232305936075 > 259.4600035150499. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.01452414231275 (T) = (0 -333.43775101894914) / Math.Sqrt((7.955639902896538 / (44)) + (46.89663269984601 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35478128813697846 = (246.1192473934111 - 333.43775101894914) / 246.1192473934111 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 317.3332033752694 > 284.50457220922067. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.27413862892172 (T) = (0 -317.97325256403144) / Math.Sqrt((59.82263498303023 / (44)) + (1.501959030458091 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17540442799531306 = (270.5224218921348 - 317.97325256403144) / 270.5224218921348 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Int16>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 315.02248454187753 > 257.4069527532303. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -100.01027814680587 (T) = (0 -315.16447562711477) / Math.Sqrt((16.722031025034028 / (44)) + (1.017813859111176 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26963557557983575 = (248.23223426389958 - 315.16447562711477) / 248.23223426389958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20SourceGen).html>) 979.55 μs 1.07 ms 1.10 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 840.56 μs 887.59 μs 1.06 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 833.50 μs 897.31 μs 1.08 0.04 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromString(Mode%3a%20Reflection).html>) 913.32 μs 983.59 μs 1.08 0.04 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(ImmutableDictionary(String%2c%20String)).DeserializeFromReader(Mode%3a%20Reflection).html>) 985.61 μs 1.07 ms 1.08 0.01 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) Test Report.html>)

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0728747062500004 > 1043.0406723632814. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.04882994494583 (T) = (0 -1077156.6767551894) / Math.Sqrt((176775724.31139025 / (44)) + (120597258.32145372 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08784196922462628 = (990177.5324249965 - 1077156.6767551894) / 990177.5324249965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 887.5882488425925 > 884.1861006167765. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.12471742268997 (T) = (0 -899629.1598843242) / Math.Sqrt((117926293.85843183 / (44)) + (68833578.40831758 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0779345205294546 = (834586.0928940737 - 899629.1598843242) / 834586.0928940737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 897.3099547511313 > 879.1943426872468. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.68394254037018 (T) = (0 -898964.5828912441) / Math.Sqrt((226862314.22978067 / (44)) + (104638653.65895973 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07355727664433155 = (837369.9312077507 - 898964.5828912441) / 837369.9312077507 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 983.5885432942708 > 965.3783318138433. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.401318042155715 (T) = (0 -985198.9041022781) / Math.Sqrt((259474841.9495172 / (44)) + (45015062.64941572 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07276173087665196 = (918376.257975927 - 985198.9041022781) / 918376.257975927 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<ImmutableDictionary<String, String>>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.066226282692308 > 1041.0987312349762. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.28979383355204 (T) = (0 -1078181.711207138) / Math.Sqrt((133601790.24639225 / (44)) + (65113799.62491411 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08854613050950952 = (990478.6586329415 - 1078181.711207138) / 990478.6586329415 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 14.26 μs 17.69 μs 1.24 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20BasicJson).html>) 37.04 μs 46.37 μs 1.25 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400B).html>) 118.87 μs 149.72 μs 1.26 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400KB).html>) 206.20 ms 307.52 ms 1.49 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 15.13 μs 18.07 μs 1.19 0.03 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400B).html>) 43.38 μs 57.01 μs 1.31 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20BasicJson).html>) 73.96 μs 91.47 μs 1.24 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 7.24 μs 8.98 μs 1.24 0.03 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400B).html>) 123.19 μs 159.75 μs 1.30 0.01 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 23.41 ms 31.24 ms 1.33 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20BasicJson).html>) 80.68 μs 100.56 μs 1.25 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20BasicJson).html>) 30.67 μs 38.56 μs 1.26 0.03 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20True%2c%20TestCase%3a%20Json400KB).html>) 200.00 ms 304.68 ms 1.52 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 7.66 μs 9.67 μs 1.26 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20True%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400KB).html>) 28.62 ms 37.60 ms 1.31 0.04 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented%3a%20False%2c%20TestRandomAccess%3a%20False%2c%20TestCase%3a%20Json400B).html>) 37.17 μs 47.92 μs 1.29 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.68726424396682 > 15.078643349288702. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.86676504402015 (T) = (0 -17482.4578002709) / Math.Sqrt((40244.71216929318 / (44)) + (22087.037163106925 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21314111077898723 = (14410.902115950048 - 17482.4578002709) / 14410.902115950048 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: BasicJson) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 46.370125026494286 > 39.7490565727104. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.07153818994049 (T) = (0 -46917.44090174304) / Math.Sqrt((749641.8131858916 / (44)) + (156269.58675159529 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24692245731536389 = (37626.59067249197 - 46917.44090174304) / 37626.59067249197 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 149.7204765873016 > 123.3405712582911. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -74.36464313120344 (T) = (0 -151130.81362524963) / Math.Sqrt((2050669.9376559393 / (44)) + (2511873.7413827735 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2947656346569713 = (116724.4554380604 - 151130.81362524963) / 116724.4554380604 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 307.52014264285714 > 217.33837203173078. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.177451191685865 (T) = (0 -310930917.38344324) / Math.Sqrt((4983465731300.487 / (44)) + (303562773254559.7 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5066440465386789 = (206373176.2640068 - 310930917.38344324) / 206373176.2640068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.07122913769795 > 15.88702050421038. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -72.94225151050378 (T) = (0 -18256.396499628394) / Math.Sqrt((40549.218932106894 / (43)) + (13788.439941445242 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.21275437091670107 = (15053.663740522068 - 18256.396499628394) / 15053.663740522068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.01236319951337 > 46.160640084371614. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -70.73603967071413 (T) = (0 -55745.79540625568) / Math.Sqrt((359288.4494159683 / (44)) + (298825.2114679458 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2700977600515953 = (43890.94852351453 - 55745.79540625568) / 43890.94852351453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 91.47166017206479 > 78.17060605129656. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -80.62130797831674 (T) = (0 -91578.52192967599) / Math.Sqrt((835793.0222451772 / (44)) + (382130.54882641986 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.23366828607698664 = (74232.69525788966 - 91578.52192967599) / 74232.69525788966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.982757183495886 > 7.638461521933598. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -63.861173052140224 (T) = (0 -9049.98401089179) / Math.Sqrt((12317.391668079264 / (44)) + (6804.69346991685 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23628320154940385 = (7320.316250798898 - 9049.98401089179) / 7320.316250798898 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 159.75282507288628 > 129.95628938494508. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -102.40845878248355 (T) = (0 -159806.63415949733) / Math.Sqrt((1885311.1468008172 / (43)) + (1194038.8678868753 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2905804931442997 = (123825.39098367526 - 159806.63415949733) / 123825.39098367526 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.237566683333334 > 24.602701986750002. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -71.42774877300687 (T) = (0 -31102264.08511905) / Math.Sqrt((335444124238.6956 / (44)) + (59151488975.8299 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3280126489671717 = (23420156.50928329 - 31102264.08511905) / 23420156.50928329 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: BasicJson) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 100.5611744021203 > 85.28717041015625. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.8948541710947 (T) = (0 -100127.9292658775) / Math.Sqrt((883316.7723349303 / (43)) + (256720.33766949712 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2341506161480128 = (81131.04507324498 - 100127.9292658775) / 81131.04507324498 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: BasicJson) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.55626562757202 > 32.18558907630968. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.276640845261085 (T) = (0 -38784.78900129904) / Math.Sqrt((386786.10897017043 / (44)) + (165034.37738862468 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2564973357386446 = (30867.386581841823 - 38784.78900129904) / 30867.386581841823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 304.6823074615384 > 211.89765292163463. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.41748107681465 (T) = (0 -301295258.2679609) / Math.Sqrt((4079723625307.3735 / (44)) + (14934979553108.984 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5011697004716075 = (200706994.1348443 - 301295258.2679609) / 200706994.1348443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.672231603434097 > 8.196440553620764. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -69.86796159312435 (T) = (0 -9642.292448596741) / Math.Sqrt((14819.95846971651 / (44)) + (5424.238173250887 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23685258308655588 = (7795.829980428611 - 9642.292448596741) / 7795.829980428611 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.59500007692308 > 30.055280601875005. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -74.29902684622837 (T) = (0 -37641014.13751672) / Math.Sqrt((477803117331.86 / (44)) + (50220084995.89656 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.30765940942735087 = (28785029.08796446 - 37641014.13751672) / 28785029.08796446 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.92121567278287 > 39.590819885947944. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.37856116927878 (T) = (0 -48163.735331308264) / Math.Sqrt((319908.1699165574 / (44)) + (66245.95991953342 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2797091611717641 = (37636.4699047768 - 48163.735331308264) / 37636.4699047768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSetReverse(String).IList(Size%3a%20512).html>) 28.48 μs 33.50 μs 1.18 0.09 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSetReverse(String).Array(Size%3a%20512).html>) 8.48 μs 14.41 μs 1.70 0.12 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSetReverse(String).Span(Size%3a%20512).html>) 7.02 μs 8.03 μs 1.14 0.16 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSetReverse(String).List(Size%3a%20512).html>) 23.06 μs 28.72 μs 1.25 0.09 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.50020618397161 > 29.963399019522146. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.979957534998256 (T) = (0 -33607.452971939136) / Math.Sqrt((130729.67025159327 / (44)) + (327606.78879402066 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.182838259940755 = (28412.551495943695 - 33607.452971939136) / 28412.551495943695 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSetReverse<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.407512349676223 > 8.61074774843503. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.542907030255098 (T) = (0 -12534.639767092645) / Math.Sqrt((30422.361509386392 / (44)) + (279438.40339418466 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5121625025852445 = (8289.214780596 - 12534.639767092645) / 8289.214780596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSetReverse<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.027868702495125 > 7.367838785699577. IsChangePoint: Marked as a change because one of 10/24/2022 9:01:44 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.373164921287245 (T) = (0 -8106.34760316794) / Math.Sqrt((139001.13200482193 / (44)) + (12112.185387623083 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12570767031983565 = (7201.11252405766 - 8106.34760316794) / 7201.11252405766 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSetReverse<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.715839179421767 > 24.5734201330862. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.949855141066225 (T) = (0 -28723.524842297636) / Math.Sqrt((886515.5837071311 / (44)) + (1664242.3520562602 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2211230369231819 = (23522.220098861806 - 28723.524842297636) / 23522.220098861806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<SByte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).DotBenchmark.html>) 336.09 ns 486.14 ns 1.45 0.11 False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).EqualsStaticBenchmark.html>) 394.30 ns 490.55 ns 1.24 0.05 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).ConditionalSelectBenchmark.html>) 348.68 ns 565.44 ns 1.62 0.07 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).SubtractBenchmark.html>) 346.92 ns 486.18 ns 1.40 0.08 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).MultiplyBenchmark.html>) 347.68 ns 486.14 ns 1.40 0.08 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanOrEqualAnyBenchmark.html>) 49.93 ns 561.27 ns 11.24 0.13 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).AddOperatorBenchmark.html>) 349.91 ns 466.75 ns 1.33 0.09 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanAllBenchmark.html>) 284.79 ns 679.19 ns 2.38 0.09 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).NegateBenchmark.html>) 349.43 ns 485.97 ns 1.39 0.08 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanOrEqualAllBenchmark.html>) 310.29 ns 734.63 ns 2.37 0.09 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanAnyBenchmark.html>) 47.63 ns 540.29 ns 11.34 0.13 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).DivisionOperatorBenchmark.html>) 390.90 ns 500.57 ns 1.28 0.08 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).MinBenchmark.html>) 434.84 ns 534.43 ns 1.23 0.06 False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).EqualsAllBenchmark.html>) 47.97 ns 52.05 ns 1.08 0.07 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).GreaterThanAllBenchmark.html>) 48.10 ns 641.16 ns 13.33 0.12 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).OnesComplementOperatorBenchmark.html>) 68.13 ns 297.97 ns 4.37 0.10 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).AddBenchmark.html>) 348.99 ns 485.99 ns 1.39 0.10 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanBenchmark.html>) 377.61 ns 477.54 ns 1.26 0.06 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).OnesComplementBenchmark.html>) 67.49 ns 299.93 ns 4.44 0.06 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).EqualsAnyBenchmark.html>) 284.78 ns 606.35 ns 2.13 0.09 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).SubtractionOperatorBenchmark.html>) 348.29 ns 468.25 ns 1.34 0.12 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).AndNotBenchmark.html>) 173.18 ns 406.22 ns 2.35 0.10 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).MaxBenchmark.html>) 426.41 ns 528.06 ns 1.24 0.10 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).GreaterThanOrEqualAllBenchmark.html>) 49.67 ns 666.46 ns 13.42 0.08 False
[GreaterThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).GreaterThanBenchmark.html>) 392.37 ns 492.33 ns 1.25 0.09 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).MultiplyOperatorBenchmark.html>) 348.71 ns 468.50 ns 1.34 0.06 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).AbsBenchmark.html>) 296.89 ns 437.62 ns 1.47 0.04 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).LessThanOrEqualBenchmark.html>) 399.34 ns 503.02 ns 1.26 0.10 False
[GreaterThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).GreaterThanOrEqualBenchmark.html>) 416.65 ns 515.69 ns 1.24 0.10 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).SquareRootBenchmark.html>) 362.80 ns 437.57 ns 1.21 0.10 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).UnaryNegateOperatorBenchmark.html>) 347.65 ns 539.18 ns 1.55 0.08 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).EqualsBenchmark.html>) 82.05 ns 87.84 ns 1.07 0.13 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).OneBenchmark.html>) 132.07 ns 230.59 ns 1.75 0.10 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).DivideBenchmark.html>) 389.78 ns 518.84 ns 1.33 0.08 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(SByte).GreaterThanAnyBenchmark.html>) 286.89 ns 608.20 ns 2.12 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;SByte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<SByte>.DotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 486.14249608932965 > 351.9003788073782. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -97.18284723445902 (T) = (0 -486.5764038482439) / Math.Sqrt((96.98080480207335 / (44)) + (1.7437919789399479 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4372774161640076 = (338.5403530147173 - 486.5764038482439) / 338.5403530147173 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 490.553699470447 > 413.05491914595945. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.16416384164293 (T) = (0 -495.6573797402634) / Math.Sqrt((26.120092412834552 / (44)) + (43.04324165357745 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25527432803456757 = (394.85980766955834 - 495.6573797402634) / 394.85980766955834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 565.4363543515756 > 366.9760420407671. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -109.10128001358012 (T) = (0 -568.8543129519484) / Math.Sqrt((39.989000153648426 / (44)) + (46.17848078865979 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6206882395483795 = (350.9955209587164 - 568.8543129519484) / 350.9955209587164 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 486.17582616970407 > 368.10293834004597. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -83.9731413507234 (T) = (0 -488.2421159487381) / Math.Sqrt((70.10838952560077 / (44)) + (15.829123794314876 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.38871383380377905 = (351.5786363353281 - 488.2421159487381) / 351.5786363353281 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 486.139869539892 > 366.756459603599. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.317919287728564 (T) = (0 -489.25581999001577) / Math.Sqrt((484.0125757738506 / (44)) + (78.79687699943211 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.37844596089516513 = (354.9328982561581 - 489.25581999001577) / 354.9328982561581 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 561.2717722378814 > 52.4322300007929. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -582.5358705856427 (T) = (0 -561.0982785088195) / Math.Sqrt((4.291452654796391 / (44)) + (10.054856023106078 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -10.08078906117048 = (50.637032743004845 - 561.0982785088195) / 50.637032743004845 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 466.7474415369366 > 367.6356359528838. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -90.1696822577041 (T) = (0 -469.49863751232647) / Math.Sqrt((67.10776851787956 / (44)) + (2.1286823427670414 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3297406643562681 = (353.075340250358 - 469.49863751232647) / 353.075340250358 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 679.1855634227541 > 299.27660128259936. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -181.2496370639127 (T) = (0 -682.2023390078634) / Math.Sqrt((19.992856161857368 / (44)) + (64.6763727431846 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.3810992959591375 = (286.507303649873 - 682.2023390078634) / 286.507303649873 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 485.96570308893536 > 366.0811488463431. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -71.08303904090187 (T) = (0 -488.5776498133663) / Math.Sqrt((34.21775858882785 / (44)) + (45.59653557187634 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39715929448533344 = (349.69359023112816 - 488.5776498133663) / 349.69359023112816 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 734.6252439532616 > 325.7596514651784. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -132.23599451720065 (T) = (0 -709.7057838691134) / Math.Sqrt((104.76930365429538 / (44)) + (98.50683219396645 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.2592011958835105 = (314.1401417289739 - 709.7057838691134) / 314.1401417289739 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 540.2921970934582 > 50.03503446561727. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -591.9974266307344 (T) = (0 -537.2304333124898) / Math.Sqrt((1.2743833015325152 / (44)) + (9.80112797787908 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -10.144418225532672 = (48.20623404832887 - 537.2304333124898) / 48.20623404832887 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 500.5655030638492 > 410.24367280104155. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -90.32882027663442 (T) = (0 -501.19710154438366) / Math.Sqrt((62.82724405627115 / (44)) + (0.9446113792018128 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28214656412122885 = (390.90468716257834 - 501.19710154438366) / 390.90468716257834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 534.4294560057903 > 458.5740387537493. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -151.71903419036624 (T) = (0 -536.3833396425238) / Math.Sqrt((9.08540590892247 / (44)) + (3.1810084946060835 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22398345883845974 = (438.2276049314774 - 536.3833396425238) / 438.2276049314774 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 52.05125086062313 > 50.50537242714985. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.287312091610364 (T) = (0 -51.70203572058934) / Math.Sqrt((0.9119546273679232 / (44)) + (0.610784426032304 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07353663453810459 = (48.16047637054738 - 51.70203572058934) / 48.16047637054738 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 641.1551288200315 > 49.95062174657354. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -150.98040047595399 (T) = (0 -645.4812218151822) / Math.Sqrt((9.212399824015735 / (44)) + (231.31763642691317 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -12.288934861679138 = (48.57283360433461 - 645.4812218151822) / 48.57283360433461 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 297.96776826956216 > 70.86564239282117. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -375.0645024390532 (T) = (0 -296.18067808436166) / Math.Sqrt((5.8138600926456965 / (44)) + (3.574544234561195 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.3617927716198373 = (67.9034272355789 - 296.18067808436166) / 67.9034272355789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 485.9915794503212 > 366.51850557558885. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.25137009695182 (T) = (0 -487.5403063930341) / Math.Sqrt((1106.4246528267001 / (43)) + (7.478132691156362 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.36135511417116656 = (358.1286773141945 - 487.5403063930341) / 358.1286773141945 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 477.5417371659683 > 397.93156652488324. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.69772774313182 (T) = (0 -485.0710715990372) / Math.Sqrt((7.860418664859266 / (44)) + (250.67254533740106 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27832102860208796 = (379.4595103621883 - 485.0710715990372) / 379.4595103621883 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 299.9276195560666 > 70.8599815937822. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -375.0368718617147 (T) = (0 -296.28251136240186) / Math.Sqrt((10.513287478523706 / (44)) + (1.9811642282270154 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.3671464723175633 = (67.84350221373964 - 296.28251136240186) / 67.84350221373964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 606.3483041903283 > 301.401917255537. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -179.1211715555724 (T) = (0 -606.6533390384517) / Math.Sqrt((120.47064489825328 / (44)) + (5.3625816468640854 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -1.090386957012376 = (290.211023850576 - 606.6533390384517) / 290.211023850576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.24942667423744 > 368.1515625738476. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.61888512608569 (T) = (0 -473.8376262880924) / Math.Sqrt((60.25400099139782 / (44)) + (102.63262490085428 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3472413299167413 = (351.70953842202516 - 473.8376262880924) / 351.70953842202516 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 406.21554939771227 > 181.84970384402584. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -211.29658917497056 (T) = (0 -409.3095390843893) / Math.Sqrt((16.266703084260655 / (43)) + (13.32227920137999 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -1.3863153200081604 = (171.5236606212584 - 409.3095390843893) / 171.5236606212584 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 528.0634546370612 > 449.22408835367844. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.459830790283554 (T) = (0 -528.2986215424187) / Math.Sqrt((164.6858127898224 / (44)) + (26.558235401440324 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22589805133898022 = (430.9482513373665 - 528.2986215424187) / 430.9482513373665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 666.456620416515 > 52.148592061566156. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -186.8468398407868 (T) = (0 -669.8424552863031) / Math.Sqrt((0.4940346278436042 / (44)) + (153.92034378134582 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -12.400706095810818 = (49.98560900426747 - 669.8424552863031) / 49.98560900426747 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 492.32578299974494 > 412.17428817118736. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.60013261212565 (T) = (0 -499.18125361961023) / Math.Sqrt((48.98861687277509 / (44)) + (250.67792845102758 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26272824392946204 = (395.31962322012913 - 499.18125361961023) / 395.31962322012913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 468.49721910352997 > 366.3255328063882. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.842174243770103 (T) = (0 -470.1992181894467) / Math.Sqrt((1478.2304065655903 / (44)) + (39.528002671016125 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3178838669381049 = (356.7834996583427 - 470.1992181894467) / 356.7834996583427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 437.6187960264103 > 312.78164783619144. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -103.9961997029849 (T) = (0 -439.97681660894233) / Math.Sqrt((54.66519135424479 / (44)) + (8.816798717420582 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4700854940903024 = (299.2865505969791 - 439.97681660894233) / 299.2865505969791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 503.0161930668017 > 423.43069738165826. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.031393947840456 (T) = (0 -507.4663925353669) / Math.Sqrt((4.04233874201942 / (44)) + (282.7753092137411 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2596229081655048 = (402.8716763133762 - 507.4663925353669) / 402.8716763133762 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 515.6859780036204 > 439.7790029973717. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -68.3675485493151 (T) = (0 -516.1902119140157) / Math.Sqrt((76.67899430969999 / (44)) + (3.0647146311308693 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22670778138727996 = (420.7931340667439 - 516.1902119140157) / 420.7931340667439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 437.57237031429855 > 381.05894808536044. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.1041207245219 (T) = (0 -438.5483720254984) / Math.Sqrt((20.71059375216724 / (44)) + (7.9746762501676685 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20693719098978858 = (363.3564159754265 - 438.5483720254984) / 363.3564159754265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 539.1789161478897 > 366.1489251154079. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.26921869435478 (T) = (0 -488.6878908894518) / Math.Sqrt((9.130821566279963 / (44)) + (195.74076068883966 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39885077512359185 = (349.3495514889893 - 488.6878908894518) / 349.3495514889893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.84419473995007 > 85.28964156688338. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/12/2022 11:17:15 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.659938271563572 (T) = (0 -89.61912641473542) / Math.Sqrt((5.719897238940539 / (44)) + (3.910122168787727 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10530167143964747 = (81.0811461978585 - 89.61912641473542) / 81.0811461978585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 230.5915351863399 > 139.03262655024855. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -106.89965833221312 (T) = (0 -230.5447779210458) / Math.Sqrt((24.770181181419296 / (44)) + (3.4756497172934817 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.7171171422938067 = (134.26269661082827 - 230.5447779210458) / 134.26269661082827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 518.8417754077065 > 409.0446482547549. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.118634066448074 (T) = (0 -527.8659129064362) / Math.Sqrt((100.84048601304852 / (44)) + (808.546025183241 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3464499383378476 = (392.0427324301941 - 527.8659129064362) / 392.0427324301941 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<SByte>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 608.196664823462 > 300.44830747767725. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.956112100819471 (T) = (0 -645.8890286329329) / Math.Sqrt((7.232152415183111 / (44)) + (9935.971732798711 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.253195567797305 = (286.65466853565033 - 645.8890286329329) / 286.65466853565033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Append_Strings - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat%3a%201).html>) 3.85 μs 4.61 μs 1.20 0.02 False
[ctor_string - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.ctor_string(length%3a%20100).html>) 468.23 ns 568.63 ns 1.21 0.04 False
[Append_Primitives - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Primitives.html>) 41.03 μs 44.79 μs 1.09 0.01 False
[Append_Strings - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat%3a%201000).html>) 2.18 ms 3.22 ms 1.48 0.02 False
[AppendLine_Strings - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.AppendLine_Strings.html>) 5.03 μs 5.75 μs 1.14 0.02 False
[Insert_Strings - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Insert_Strings.html>) 12.89 μs 14.45 μs 1.12 0.04 False
[Insert_Primitives - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Insert_Primitives.html>) 628.38 μs 693.27 μs 1.10 0.00 False
[Append_Char_Capacity - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length%3a%20100).html>) 4.44 μs 5.01 μs 1.13 0.02 False
[Append_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Char(length%3a%20100000).html>) 4.32 ms 4.73 ms 1.10 0.05 False
[ToString_MultipleSegments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length%3a%20100).html>) 733.37 ns 904.76 ns 1.23 0.03 False
[Append_NonEmptySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_NonEmptySpan.html>) 6.01 μs 7.56 μs 1.26 0.03 False
[Append_ValueTypes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes.html>) 110.66 μs 125.64 μs 1.14 0.01 False
[Append_Memory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Memory.html>) 7.11 μs 9.07 μs 1.28 0.04 False
[Append_ValueTypes_Interpolated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated.html>) 114.09 μs 133.13 μs 1.17 0.01 False
[ToString_SingleSegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length%3a%20100).html>) 350.04 ns 434.10 ns 1.24 0.07 False
[Append_Char - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Char(length%3a%20100).html>) 5.30 μs 5.71 μs 1.08 0.04 False
[Append_Char_Capacity - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length%3a%20100000).html>) 4.30 ms 4.62 ms 1.07 0.09 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.60567552906239 > 4.103797176140319. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.798473891983992 (T) = (0 -4666.509523756999) / Math.Sqrt((139165.22545327013 / (44)) + (2702.52198885282 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17121389343226137 = (3984.3358671930687 - 4666.509523756999) / 3984.3358671930687 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.ctor_string(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 568.6299549094825 > 498.1687150955172. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.589901600607792 (T) = (0 -562.9226308793835) / Math.Sqrt((2111.90399049326 / (44)) + (55.369325992158956 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.191604273402875 = (472.40736160826305 - 562.9226308793835) / 472.40736160826305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Primitives ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 44.78958677696583 > 43.25570214346385. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.69273003704715 (T) = (0 -44892.263494035586) / Math.Sqrt((224367.33564308684 / (44)) + (38643.349208314896 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09790210241739204 = (40889.13154933443 - 44892.263494035586) / 40889.13154933443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2217145825000006 > 2.2874112906129804. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.883575108968362 (T) = (0 -3212019.1662661787) / Math.Sqrt((266449823417.52335 / (44)) + (429414512.9225997 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3592517085346977 = (2363078.9986122614 - 3212019.1662661787) / 2363078.9986122614 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.AppendLine_Strings ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.747158568833886 > 5.291737625565146. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.915890393215106 (T) = (0 -5744.709261157088) / Math.Sqrt((131185.21188585562 / (44)) + (2246.191739568396 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13130987362138302 = (5077.927272717923 - 5744.709261157088) / 5077.927272717923 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Insert_Strings ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.454465174418607 > 13.687839927513997. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.632188176503412 (T) = (0 -14641.085372392086) / Math.Sqrt((431590.4427134558 / (44)) + (20729.38891952036 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.10979239266522946 = (13192.634468533965 - 14641.085372392086) / 13192.634468533965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Insert_Primitives ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 693.2694124611801 > 662.7466734891827. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.08753181690031 (T) = (0 -694970.722882335) / Math.Sqrt((35637834.97492338 / (44)) + (26459220.59356683 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1047897800308798 = (629052.4545428995 - 694970.722882335) / 629052.4545428995 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.0118482871098 > 4.686820628593289. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 11/24/2022 6:35:15 AM, 11/24/2022 5:37:24 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.260289788927842 (T) = (0 -4802.046167232264) / Math.Sqrt((1939.5979480126668 / (44)) + (3951.562602504107 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07144774088018142 = (4481.8295694827275 - 4802.046167232264) / 4481.8295694827275 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.728078122916667 > 4.542199982496995. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.11602213360986 (T) = (0 -4662324.845274153) / Math.Sqrt((9220457936.637732 / (43)) + (1261414591.2360442 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.0635168585873527 = (4383874.884190385 - 4662324.845274153) / 4383874.884190385 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 904.7630108556834 > 751.3555256271475. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.150193046994431 (T) = (0 -914.0586305903929) / Math.Sqrt((18065.240761554658 / (44)) + (78.74454641870545 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29272585979353005 = (707.0784758157336 - 914.0586305903929) / 707.0784758157336 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_NonEmptySpan ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.555508268482489 > 6.233496743148828. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.468761632212702 (T) = (0 -7665.640488004891) / Math.Sqrt((576731.3835766243 / (44)) + (13915.5437706977 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26287510269438874 = (6069.990984579533 - 7665.640488004891) / 6069.990984579533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 125.63810542028536 > 117.270780456816. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.195852702265704 (T) = (0 -127708.65477294535) / Math.Sqrt((2690719.931098841 / (44)) + (1586132.345716537 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14703833791480295 = (111337.73872380453 - 127708.65477294535) / 111337.73872380453 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Memory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.072904535135002 > 7.5100252650206425. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.085423888145442 (T) = (0 -8993.847906880133) / Math.Sqrt((570083.6700191639 / (44)) + (6344.984204584667 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2212280669362208 = (7364.593191379577 - 8993.847906880133) / 7364.593191379577 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 133.13165886752137 > 120.85704833580135. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/6/2022 2:12:43 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.62618768171094 (T) = (0 -137489.21598629496) / Math.Sqrt((1176447.740203267 / (23)) + (28768456.07065777 / (36))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (36) - 2, .025) and -0.1882897781222781 = (115703.44079165088 - 137489.21598629496) / 115703.44079165088 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.ToString_SingleSegment(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 434.09951630327475 > 376.8625390938425. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.555947861853936 (T) = (0 -434.45267165098943) / Math.Sqrt((3893.5278416783644 / (44)) + (35.17942969561916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23106371860559877 = (352.9083548519205 - 434.45267165098943) / 352.9083548519205 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.709490609948128 > 5.574926668829498. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.516582794136463 (T) = (0 -5715.343724311852) / Math.Sqrt((18634.606920599024 / (43)) + (2787.468466711217 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.06252858656844076 = (5379.002312559154 - 5715.343724311852) / 5379.002312559154 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Tests.Perf_StringBuilder.Append_Char_Capacity(length: 100000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.621562503348215 > 4.520270486694712. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -3.8707659406465984 (T) = (0 -4761217.138911783) / Math.Sqrt((6681084680.736915 / (44)) + (160433443777.24222 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09250502122816158 = (4358073.460897566 - 4761217.138911783) / 4358073.460897566 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%22Bogus%22).html>) 495.76 ns 662.66 ns 1.34 0.08 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.Parse(value%3a%20%22%20True%20%22).html>) 725.70 ns 1.03 μs 1.41 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.Parse(value%3a%20%22false%22).html>) 235.36 ns 341.71 ns 1.45 0.16 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%22False%22).html>) 193.95 ns 303.63 ns 1.57 0.16 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.Parse(value%3a%20%22true%22).html>) 188.73 ns 294.58 ns 1.56 0.15 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%22TRUE%22).html>) 147.07 ns 248.15 ns 1.69 0.17 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%22%20True%20%22).html>) 687.55 ns 976.03 ns 1.42 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.Parse(value%3a%20%22TRUE%22).html>) 189.34 ns 294.69 ns 1.56 0.12 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.Parse(value%3a%20%22False%22).html>) 237.08 ns 346.16 ns 1.46 0.10 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%220%22).html>) 191.77 ns 202.73 ns 1.06 0.09 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Boolean.TryParse(value%3a%20%22true%22).html>) 145.56 ns 250.82 ns 1.72 0.14 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Boolean.TryParse(value: "Bogus") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 662.6599135196567 > 519.3191029419205. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.42136219675743 (T) = (0 -678.3434145557809) / Math.Sqrt((156.0140959468617 / (44)) + (324.7356685611771 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3972359199369699 = (485.4895332109565 - 678.3434145557809) / 485.4895332109565 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.Parse(value: " True ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0252970373838024 > 767.9970556765294. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.084986219065186 (T) = (0 -1011.6489819380091) / Math.Sqrt((149.32952076876026 / (44)) + (358.9528640062786 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3779661354283658 = (734.1609898297825 - 1011.6489819380091) / 734.1609898297825 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.Parse(value: "false") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 341.7124348531392 > 249.4722987996866. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.54929850073788 (T) = (0 -348.13101045601) / Math.Sqrt((19.716221768716686 / (44)) + (51.48341346792246 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.45840289807101353 = (238.70702047868434 - 348.13101045601) / 238.70702047868434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: "False") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 303.63142961979787 > 203.60845672246828. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -78.50547702966901 (T) = (0 -302.2456726901097) / Math.Sqrt((7.893992744888338 / (44)) + (25.23254037634373 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5489208812325613 = (195.13306092793857 - 302.2456726901097) / 195.13306092793857 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.Parse(value: "true") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.5757894054864 > 198.78268116305742. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -85.52276813623567 (T) = (0 -293.6441167505057) / Math.Sqrt((39.58789812932613 / (44)) + (6.721482273819502 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.5200247748925426 = (193.18377016010461 - 293.6441167505057) / 193.18377016010461 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: "TRUE") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 248.15136630471153 > 154.68169275929893. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -112.27755263985661 (T) = (0 -250.09627086043142) / Math.Sqrt((8.448661245387894 / (44)) + (9.56831593257182 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6919827173285867 = (147.8125445957863 - 250.09627086043142) / 147.8125445957863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: " True ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 976.0289643048022 > 726.8191370444374. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -75.15301004530677 (T) = (0 -963.2557157300483) / Math.Sqrt((155.38762572955838 / (44)) + (145.2443159821451 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3958984933630301 = (690.0614337718432 - 963.2557157300483) / 690.0614337718432 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.Parse(value: "TRUE") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.68637045913664 > 198.63666702287426. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.55113347574672 (T) = (0 -294.09020382474046) / Math.Sqrt((44.340127670060845 / (44)) + (8.232298185141035 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5289679713473833 = (192.34556206274044 - 294.09020382474046) / 192.34556206274044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.Parse(value: "False") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 346.16407738458054 > 248.52609512315456. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.86059159174546 (T) = (0 -345.86175056364317) / Math.Sqrt((15.912238573092065 / (44)) + (26.194614514307556 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4493932934238147 = (238.62519036957508 - 345.86175056364317) / 238.62519036957508 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: "0") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 202.7331712871136 > 201.68236800342493. IsChangePoint: Marked as a change because one of 10/13/2022 11:37:03 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.78888435331844 (T) = (0 -206.72632875641924) / Math.Sqrt((7.4176172146838715 / (44)) + (50.77158167804539 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07645286012288997 = (192.0440145728434 - 206.72632875641924) / 192.0440145728434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Boolean.TryParse(value: "true") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 250.81848371440006 > 157.01814703778712. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -83.52341765291605 (T) = (0 -251.11016546986542) / Math.Sqrt((6.5682562042214085 / (44)) + (19.068171440576847 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.6917634693840478 = (148.4310129720982 - 251.11016546986542) / 148.4310129720982 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 22.99 μs 27.32 μs 1.19 0.03 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 18.04 μs 21.08 μs 1.17 0.02 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 68.71 μs 77.93 μs 1.13 0.04 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 41.22 μs 48.11 μs 1.17 0.07 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 3.02 μs 3.49 μs 1.16 0.04 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 17.47 μs 20.12 μs 1.15 0.03 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 183.33 μs 210.64 μs 1.15 0.03 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json400B).html>) 25.64 μs 29.05 μs 1.13 0.02 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 23.82 μs 27.71 μs 1.16 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 2.05 ms 2.31 ms 1.13 0.04 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 51.26 μs 59.03 μs 1.15 0.04 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 51.45 μs 60.84 μs 1.18 0.04 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 2.81 μs 3.12 μs 1.11 0.05 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 28.66 μs 32.70 μs 1.14 0.03 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 212.77 μs 239.89 μs 1.13 0.01 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 66.28 μs 78.25 μs 1.18 0.05 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 184.05 μs 216.02 μs 1.17 0.05 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 51.80 μs 60.71 μs 1.17 0.04 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 1.30 ms 1.45 ms 1.12 0.02 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 156.99 μs 175.84 μs 1.12 0.03 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 179.80 μs 208.48 μs 1.16 0.04 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 1.77 ms 2.04 ms 1.15 0.03 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 1.67 ms 1.96 ms 1.17 0.05 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json4KB).html>) 221.73 μs 262.36 μs 1.18 0.02 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 2.19 μs 2.41 μs 1.10 0.03 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 49.57 μs 55.73 μs 1.12 0.04 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 88.52 μs 98.82 μs 1.12 0.05 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 180.99 μs 215.28 μs 1.19 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 82.55 μs 101.25 μs 1.23 0.04 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 37.61 μs 43.22 μs 1.15 0.04 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 1.79 ms 2.04 ms 1.14 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 28.87 μs 33.91 μs 1.17 0.02 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json40KB).html>) 1.48 ms 1.69 ms 1.14 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 2.69 μs 3.01 μs 1.12 0.04 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfStrings).html>) 36.40 μs 40.70 μs 1.12 0.02 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 2.52 μs 2.77 μs 1.10 0.04 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 5.42 μs 6.09 μs 1.12 0.05 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfStrings).html>) 40.57 μs 47.57 μs 1.17 0.03 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json40KB).html>) 2.20 ms 2.60 ms 1.18 0.01 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 88.58 μs 101.31 μs 1.14 0.03 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 23.12 μs 26.59 μs 1.15 0.03 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 204.76 μs 241.00 μs 1.18 0.03 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 132.40 μs 147.82 μs 1.12 0.02 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 180.44 μs 208.25 μs 1.15 0.04 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20DeepTree).html>) 207.63 μs 246.02 μs 1.18 0.03 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 68.09 μs 78.19 μs 1.15 0.05 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 218.42 μs 245.85 μs 1.13 0.05 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 270.72 μs 322.12 μs 1.19 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 325.23 μs 364.40 μs 1.12 0.02 False
[ReadSpanEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20BroadTree).html>) 180.45 μs 204.59 μs 1.13 0.04 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20HelloWorld).html>) 2.44 μs 2.72 μs 1.11 0.02 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20HelloWorld).html>) 6.04 μs 6.77 μs 1.12 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 326.50 μs 387.06 μs 1.19 0.05 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20BroadTree).html>) 272.67 μs 317.61 μs 1.16 0.02 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 113.04 μs 130.70 μs 1.16 0.03 False
[ReadReturnBytes - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 170.96 μs 231.00 μs 1.35 0.03 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20DeepTree).html>) 110.34 μs 122.62 μs 1.11 0.05 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20Json400B).html>) 32.43 μs 36.76 μs 1.13 0.01 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20False%2c%20TestCase%3a%20LotsOfNumbers).html>) 86.47 μs 96.59 μs 1.12 0.04 False
[ReadMultiSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20LotsOfNumbers).html>) 68.54 μs 78.76 μs 1.15 0.03 False
[ReadSingleSpanSequenceEmptyLoop - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact%3a%20True%2c%20TestCase%3a%20Json4KB).html>) 130.92 μs 148.75 μs 1.14 0.04 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Reader*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.323908919156416 > 24.05959944974099. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.89933767049491 (T) = (0 -27275.852463308278) / Math.Sqrt((374566.2867420395 / (44)) + (69697.04813601713 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17202556439154817 = (23272.404025989326 - 27275.852463308278) / 23272.404025989326 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.083248828828822 > 19.139886346098876. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.95600799424111 (T) = (0 -21058.163489552513) / Math.Sqrt((78610.33795956787 / (44)) + (52488.584991836084 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15539677920963266 = (18225.915000349647 - 21058.163489552513) / 18225.915000349647 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 77.92987151741292 > 71.08508448631976. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.23983061511603 (T) = (0 -78781.42738226548) / Math.Sqrt((774171.0132669194 / (44)) + (950182.2632949178 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1602191807732744 = (67902.19355773661 - 78781.42738226548) / 67902.19355773661 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.11069229487179 > 43.56194013412416. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.28982125843039 (T) = (0 -48307.64913231472) / Math.Sqrt((339823.08113110665 / (44)) + (411770.10601247044 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1587602101409274 = (41689.08175267736 - 48307.64913231472) / 41689.08175267736 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.492491676106328 > 3.1680198645198168. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.682319266680366 (T) = (0 -3397.074895508576) / Math.Sqrt((3290.4949754538925 / (44)) + (2547.2434468757137 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1166029439348201 = (3042.3302338229146 - 3397.074895508576) / 3042.3302338229146 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.122481684981686 > 18.339730342325037. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.68745894544136 (T) = (0 -20275.2229306433) / Math.Sqrt((27957.996576091256 / (43)) + (62464.613954871245 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.15326922835880546 = (17580.650235068326 - 20275.2229306433) / 17580.650235068326 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 210.6448764076577 > 192.75554712027616. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.950420097980874 (T) = (0 -210318.75619644288) / Math.Sqrt((7823120.826113323 / (44)) + (1938949.3009357913 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14460477712377975 = (183747.9280183876 - 210318.75619644288) / 183747.9280183876 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.049433488516446 > 27.00413503372679. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.63790554615148 (T) = (0 -28886.71166330256) / Math.Sqrt((116170.59152911273 / (43)) + (79043.75964210507 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.12769925948903058 = (25615.61641566685 - 28886.71166330256) / 25615.61641566685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.713106927175843 > 24.93301025683301. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.55422407677974 (T) = (0 -27534.453373709675) / Math.Sqrt((62283.90324633755 / (44)) + (81779.50220026264 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15486089047582485 = (23842.22515524355 - 27534.453373709675) / 23842.22515524355 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3123797589285715 > 2.157110157701322. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.482704688555465 (T) = (0 -2333575.211388889) / Math.Sqrt((848756420.0733863 / (44)) + (1575249929.2086544 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13862861069399418 = (2049461.246162236 - 2333575.211388889) / 2049461.246162236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.03044050314466 > 53.603153183220826. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.34284575733619 (T) = (0 -58828.61096097241) / Math.Sqrt((593892.9920789835 / (44)) + (162743.192367834 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15410561681626428 = (50973.33389924748 - 58828.61096097241) / 50973.33389924748 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.83987836286088 > 54.17749678446459. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.33032964200206 (T) = (0 -60688.38501149208) / Math.Sqrt((349001.1443357032 / (44)) + (767855.141534916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17586954113777611 = (51611.49505817606 - 60688.38501149208) / 51611.49505817606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.121336103909054 > 2.9471099699484156. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.532219054768976 (T) = (0 -3131.610724683768) / Math.Sqrt((2250.99904911095 / (44)) + (1970.1359182742106 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10766589451754975 = (2827.215986502644 - 3131.610724683768) / 2827.215986502644 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.69635202331961 > 29.98504213731537. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.972218554468633 (T) = (0 -32307.74954868732) / Math.Sqrt((497180.1643526277 / (44)) + (195842.40202445956 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13096357776644915 = (28566.56941374912 - 32307.74954868732) / 28566.56941374912 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 239.89324476010103 > 226.46197527883194. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.64595056679842 (T) = (0 -243688.5807123696) / Math.Sqrt((3523240.066778062 / (44)) + (6317038.287170996 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1301720947858519 = (215620.77301027716 - 243688.5807123696) / 215620.77301027716 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.24556247916668 > 68.73221088127616. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.652330756066995 (T) = (0 -76889.06959651144) / Math.Sqrt((488514.854079473 / (44)) + (2190743.7952790135 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18047215560078175 = (65134.16621621204 - 76889.06959651144) / 65134.16621621204 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.02345901826484 > 193.75467204779414. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 5:13:36 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.770807941338575 (T) = (0 -216204.5792309317) / Math.Sqrt((2545617.889909989 / (44)) + (10191601.170117963 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17648052272576153 = (183772.34051441168 - 216204.5792309317) / 183772.34051441168 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.70589199463328 > 54.62799592913828. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.140279762802052 (T) = (0 -60993.565649082375) / Math.Sqrt((1506975.4917762734 / (44)) + (644358.1360279873 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.16911673559872567 = (52170.63770611963 - 60993.565649082375) / 52170.63770611963 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4544817382305195 > 1.3465146112369795. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.09626390529213 (T) = (0 -1453346.139747362) / Math.Sqrt((175874781.13550997 / (42)) + (91667236.7389833 / (16))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (16) - 2, .025) and -0.13277814462775347 = (1282992.7436717553 - 1453346.139747362) / 1282992.7436717553 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 175.84241502808993 > 166.33721292929295. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.47245261195358 (T) = (0 -176969.66923699022) / Math.Sqrt((2144202.395258425 / (44)) + (4273594.964609123 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1144836119517069 = (158790.73262196942 - 176969.66923699022) / 158790.73262196942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.47697346491228 > 190.6961211241981. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.57483145503107 (T) = (0 -210107.40564892365) / Math.Sqrt((3619045.654507674 / (44)) + (6516936.947977586 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.15254719165419234 = (182298.31036017466 - 210107.40564892365) / 182298.31036017466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.03639739375 > 1.8661748. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.73449620542072 (T) = (0 -2045235.1684077382) / Math.Sqrt((541093582.2239277 / (44)) + (447011698.1954669 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14423561696313708 = (1787424.8433516712 - 2045235.1684077382) / 1787424.8433516712 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.9612140645833334 > 1.7626225316875002. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.593338120373815 (T) = (0 -1945039.6044703694) / Math.Sqrt((977607814.3475258 / (44)) + (229811068.04417008 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15917828722297214 = (1677946.8921300059 - 1945039.6044703694) / 1677946.8921300059 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 262.35805631944436 > 233.59593716769365. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.87468397229731 (T) = (0 -263998.71776599326) / Math.Sqrt((11401196.780196989 / (44)) + (64962940.66484165 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18087927431328435 = (223561.1408452538 - 263998.71776599326) / 223561.1408452538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.41397873971311 > 2.2539295333676344. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.051150265935483 (T) = (0 -2379.9198058507927) / Math.Sqrt((979.0498365567105 / (44)) + (1228.4389875955683 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10443142361251419 = (2154.882372022927 - 2379.9198058507927) / 2154.882372022927 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.726156331553966 > 52.12349061227844. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.369058782525045 (T) = (0 -56109.46173920153) / Math.Sqrt((335554.4193788109 / (44)) + (307456.2772052182 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1333045848235794 = (49509.604470484024 - 56109.46173920153) / 49509.604470484024 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 98.82445007861635 > 92.97479428976916. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.11099991594512 (T) = (0 -99652.72588884513) / Math.Sqrt((4668195.229772854 / (44)) + (1084098.2859847795 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12981431475323665 = (88202.74675897547 - 99652.72588884513) / 88202.74675897547 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.2819230063601 > 191.8172140170785. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.14877686082566 (T) = (0 -216464.44194572663) / Math.Sqrt((4599289.892692124 / (44)) + (32704147.37565286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18296086563969657 = (182985.2941320012 - 216464.44194572663) / 182985.2941320012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 101.25101436965814 > 86.47820967130235. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.096048248040006 (T) = (0 -96701.24083548803) / Math.Sqrt((2072001.9615628535 / (44)) + (2547950.1883390276 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1761678190670587 = (82217.21362194033 - 96701.24083548803) / 82217.21362194033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.21743072916667 > 39.720191585490866. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.50485568877027 (T) = (0 -43685.09545251378) / Math.Sqrt((206391.66343355022 / (44)) + (295061.11404115066 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14999030455710877 = (37987.359788514084 - 43685.09545251378) / 37987.359788514084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0403401031249997 > 1.8685800706770836. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.770757094715115 (T) = (0 -2048819.640456349) / Math.Sqrt((215255295.81061327 / (44)) + (523442684.2165152 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.151649695428812 = (1779030.2455587238 - 2048819.640456349) / 1779030.2455587238 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.91399200513539 > 30.573198920622524. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.546271918969055 (T) = (0 -34095.10656734657) / Math.Sqrt((203123.35045580115 / (44)) + (192768.5973522262 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1810504232078732 = (28868.459718036604 - 34095.10656734657) / 28868.459718036604 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6865687545833337 > 1.5567639881107955. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.23325319560177 (T) = (0 -1670188.8294557813) / Math.Sqrt((164172693.72457153 / (44)) + (1875606560.7597039 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12396621773391525 = (1485977.7839436606 - 1670188.8294557813) / 1485977.7839436606 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.01099825434411 > 2.831397768371907. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.770308543473327 (T) = (0 -3003.6472279507566) / Math.Sqrt((2282.4224763885977 / (44)) + (1210.6165032005065 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11435896525702371 = (2695.403655013422 - 3003.6472279507566) / 2695.403655013422 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.699102562313996 > 38.29109711823569. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.258708187357367 (T) = (0 -40902.646033104604) / Math.Sqrt((158632.63267532256 / (43)) + (220102.31374032932 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.11552210966072325 = (36666.817877365786 - 40902.646033104604) / 36666.817877365786 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.774106456348493 > 2.6132438676399294. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.969034200560206 (T) = (0 -2743.964744613662) / Math.Sqrt((1201.0062894344148 / (44)) + (2759.6806717829277 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10519857831650725 = (2482.779835632257 - 2743.964744613662) / 2482.779835632257 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.092057210172993 > 5.6852359846945095. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.024332826212888 (T) = (0 -5955.531426066866) / Math.Sqrt((11258.062493170013 / (44)) + (11754.116759537214 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1015192099279066 = (5406.65235103494 - 5955.531426066866) / 5406.65235103494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: LotsOfStrings) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.57480923526423 > 43.76021368608027. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.84523669866713 (T) = (0 -47882.33349992533) / Math.Sqrt((284541.39173239283 / (44)) + (240543.83118410275 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16771729538388092 = (41005.073479008686 - 47882.33349992533) / 41005.073479008686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: Json40KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.5977847222222223 > 2.310543380800781. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.76469737331745 (T) = (0 -2567306.823731612) / Math.Sqrt((1288847794.4780526 / (44)) + (511307213.287073 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1696114400444605 = (2195008.304325427 - 2567306.823731612) / 2195008.304325427 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 101.31431239177489 > 93.56500541370738. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.28537262440838 (T) = (0 -100597.07582704112) / Math.Sqrt((1529320.9819109177 / (44)) + (533529.1697428977 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1288048912865737 = (89118.21396555429 - 100597.07582704112) / 89118.21396555429 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.59332776121522 > 24.274941824549533. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.772118140120703 (T) = (0 -26737.159200498234) / Math.Sqrt((54932.0121702804 / (44)) + (885915.7161511421 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15698079892983083 = (23109.42344525443 - 26737.159200498234) / 23109.42344525443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 240.99553571428572 > 217.89943842231784. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 8:42:28 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.15237481196634 (T) = (0 -241026.52398316297) / Math.Sqrt((4340169.086534875 / (44)) + (2365010.0732071 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1613739276392186 = (207535.67670759524 - 241026.52398316297) / 207535.67670759524 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 147.8168536828737 > 138.58353980435925. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.28480702171128 (T) = (0 -149201.56525207974) / Math.Sqrt((1365969.3673721647 / (43)) + (1477114.2341441582 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.12914306645957976 = (132136.98926558546 - 149201.56525207974) / 132136.98926558546 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 208.25244505555554 > 189.4374995959052. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.75657977317272 (T) = (0 -207039.86984472894) / Math.Sqrt((33147903.245261226 / (44)) + (8378437.690711363 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13683185051711008 = (182120.04682183458 - 207039.86984472894) / 182120.04682183458 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 246.02156173941802 > 218.17349446570722. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 5:13:36 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.0653530581458 (T) = (0 -246584.5965537909) / Math.Sqrt((2917682.1132841143 / (44)) + (3105702.6333363457 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18332181529647873 = (208383.3775108842 - 246584.5965537909) / 208383.3775108842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.18591668750001 > 72.28285838610196. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.95942427619817 (T) = (0 -79376.01904167836) / Math.Sqrt((1029872.6437029481 / (44)) + (1049727.5092839014 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15552487668030607 = (68692.60943106374 - 79376.01904167836) / 68692.60943106374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 245.85410143229169 > 230.43645293106295. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.82724370709644 (T) = (0 -245423.8420807016) / Math.Sqrt((5043534.296476657 / (44)) + (6519813.503002112 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11377423583535982 = (220353.31235385177 - 245423.8420807016) / 220353.31235385177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 322.1205780612246 > 288.3234762993422. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.71264949195814 (T) = (0 -319479.9821154757) / Math.Sqrt((8763451.077062098 / (43)) + (11045959.045023087 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.16444461112033834 = (274362.5407893784 - 319479.9821154757) / 274362.5407893784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 364.40012416943523 > 339.7353346396684. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.73331881709932 (T) = (0 -368804.2915226621) / Math.Sqrt((18155531.91025122 / (44)) + (14618575.362593504 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14115731861772463 = (323184.44223745767 - 368804.2915226621) / 323184.44223745767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSpanEmptyLoop(IsDataCompact: True, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 204.5933979166667 > 189.5122840247845. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.18421945596352 (T) = (0 -206632.55233867222) / Math.Sqrt((5508724.610032587 / (44)) + (8047049.765845775 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14507719914039338 = (180452.94456460295 - 206632.55233867222) / 180452.94456460295 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.717716556650032 > 2.605589739093011. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.873154083369695 (T) = (0 -2719.9354026421947) / Math.Sqrt((1247.834867270473 / (44)) + (1748.13283285328 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09643346953786679 = (2480.711760640264 - 2719.9354026421947) / 2480.711760640264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.765081352106621 > 6.3738617531082635. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.99963601866222 (T) = (0 -6789.033260557388) / Math.Sqrt((8620.142073278706 / (44)) + (19074.873980716446 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12008470977166158 = (6061.178410284154 - 6789.033260557388) / 6061.178410284154 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 387.0624996875 > 345.0530891894532. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.696696842355383 (T) = (0 -391301.48255908646) / Math.Sqrt((51861492.18222353 / (44)) + (91510617.4458927 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18552833191467635 = (330065.06215428753 - 391301.48255908646) / 330065.06215428753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: BroadTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 317.6143736607143 > 289.5736896929825. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.90062853179555 (T) = (0 -320400.83101030474) / Math.Sqrt((18155230.2945701 / (44)) + (17912170.36946674 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16019398759712367 = (276161.43027416174 - 320400.83101030474) / 276161.43027416174 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.69887165798613 > 118.50302805980215. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.61207880659089 (T) = (0 -129917.67547571407) / Math.Sqrt((4504423.393219001 / (44)) + (1077823.7095324427 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.15258973997784558 = (112718.05653780267 - 129917.67547571407) / 112718.05653780267 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadReturnBytes(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 231.00198400297617 > 180.6229806713599. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.875884532397766 (T) = (0 -202095.43419414878) / Math.Sqrt((12883169.6232882 / (44)) + (66120100.24549113 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17490067828275907 = (172010.654117191 - 202095.43419414878) / 172010.654117191 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: DeepTree) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 122.62287217494419 > 116.72105219696195. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.068554286120785 (T) = (0 -124617.55981935405) / Math.Sqrt((2164393.080438879 / (44)) + (17116604.71517558 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11814092044097693 = (111450.6745448569 - 124617.55981935405) / 111450.6745448569 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: Json400B) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 36.763792947692046 > 33.772890622974856. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.09344886841883 (T) = (0 -36460.022563036975) / Math.Sqrt((106972.18942895003 / (44)) + (65292.4400603184 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1304836124433531 = (32251.70375025134 - 36460.022563036975) / 32251.70375025134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.59306410455487 > 90.58351818662348. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.553550064533534 (T) = (0 -99043.72884357195) / Math.Sqrt((1745746.1241420878 / (44)) + (18586338.1892091 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15195677433456975 = (85978.68518181576 - 99043.72884357195) / 85978.68518181576 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadMultiSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: LotsOfNumbers) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.76481481481481 > 71.29244861751253. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.584708616665242 (T) = (0 -77098.86585818804) / Math.Sqrt((892911.5397664021 / (43)) + (870820.9757806042 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.12426228503397442 = (68577.29453750925 - 77098.86585818804) / 68577.29453750925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: True, TestCase: Json4KB) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 148.74543630952383 > 139.66843780394913. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.78985654732169 (T) = (0 -150010.69900462244) / Math.Sqrt((2925826.7654586667 / (44)) + (675078.5601906709 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13030398508359223 = (132717.1282984801 - 150010.69900462244) / 132717.1282984801 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.TryAddGiventSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddGiventSize(Int32).ConcurrentDictionary(Count%3a%20512).html>) 332.66 μs 367.92 μs 1.11 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddGiventSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddGiventSize<Int32>.ConcurrentDictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 367.92486706171735 > 350.34708207907124. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.789430869214296 (T) = (0 -366654.7047306033) / Math.Sqrt((32079687.800198942 / (44)) + (21557667.827450916 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09338280253930745 = (335339.7400060369 - 366654.7047306033) / 335339.7400060369 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Tests.Perf_PriorityQueue<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).HeapSort(Size%3a%2010).html>) 30.06 μs 34.45 μs 1.15 0.07 False
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).HeapSort(Size%3a%20100).html>) 661.15 μs 778.55 μs 1.18 0.05 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).Dequeue_And_Enqueue(Size%3a%2010).html>) 74.23 μs 89.47 μs 1.21 0.09 False
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).K_Max_Elements(Size%3a%2010).html>) 13.42 μs 15.23 μs 1.13 0.05 False
[K_Max_Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).K_Max_Elements(Size%3a%201000).html>) 734.08 μs 785.48 μs 1.07 0.10 False
[HeapSort - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).HeapSort(Size%3a%201000).html>) 10.78 ms 12.55 ms 1.16 0.08 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).Dequeue_And_Enqueue(Size%3a%201000).html>) 23.67 ms 27.86 ms 1.18 0.05 False
[Dequeue_And_Enqueue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Tests.Perf_PriorityQueue(String%2c%20String).Dequeue_And_Enqueue(Size%3a%20100).html>) 1.57 ms 1.86 ms 1.18 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.447176361295064 > 31.618534474896734. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.368109374944872 (T) = (0 -34782.17961718598) / Math.Sqrt((351587.41672262305 / (44)) + (337294.564779006 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14590320522237507 = (30353.505827253637 - 34782.17961718598) / 30353.505827253637 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 778.5549111607143 > 690.5780119966945. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.958901606616525 (T) = (0 -750200.166244374) / Math.Sqrt((81178278.70128778 / (44)) + (145012680.96274737 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1447573795572274 = (655335.5144428408 - 750200.166244374) / 655335.5144428408 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.Dequeue_And_Enqueue(Size: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.47452658617425 > 78.00892215436167. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.119419469745452 (T) = (0 -85821.26121636505) / Math.Sqrt((1439218.3289461632 / (44)) + (2021534.7587955014 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14852732061046575 = (74722.87308825122 - 85821.26121636505) / 74722.87308825122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 10) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.22793455333982 > 14.172091317631596. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.423436476766057 (T) = (0 -15324.27783059581) / Math.Sqrt((69002.71549342135 / (44)) + (47412.544647350216 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12878115303236684 = (13575.951183653755 - 15324.27783059581) / 13575.951183653755 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.K_Max_Elements(Size: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 785.4803573660714 > 734.4571139062499. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.531002836056153 (T) = (0 -799089.1619552528) / Math.Sqrt((167471488.31562713 / (44)) + (174555639.2352921 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1565945988723568 = (690898.2306629648 - 799089.1619552528) / 690898.2306629648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.HeapSort(Size: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.551147279017856 > 11.291500242578126. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.300476679232474 (T) = (0 -12410328.691003513) / Math.Sqrt((14911389935.19014 / (44)) + (23132344648.011326 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1545356116608455 = (10749195.231103145 - 12410328.691003513) / 10749195.231103145 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.Dequeue_And_Enqueue(Size: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.85611852592593 > 25.028563009692306. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.545038452931585 (T) = (0 -27576542.53266043) / Math.Sqrt((251671931950.04593 / (44)) + (110050932863.5002 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1558723124055075 = (23857775.843138218 - 27576542.53266043) / 23857775.843138218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Tests.Perf_PriorityQueue<String, String>.Dequeue_And_Enqueue(Size: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.856013887037037 > 1.653755610046875. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.744934369124657 (T) = (0 -1844194.1479062801) / Math.Sqrt((701686913.99125 / (44)) + (1542147421.0350044 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16321252536336367 = (1585431.8172254823 - 1844194.1479062801) / 1585431.8172254823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>) 4.21 μs 4.61 μs 1.10 0.04 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20Reflection).html>) 19.69 μs 20.96 μs 1.06 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 10.18 μs 10.93 μs 1.07 0.02 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(SimpleStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 18.47 μs 20.23 μs 1.10 0.01 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.614283951851851 > 4.471017302381975. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.376863124779204 (T) = (0 -4689.276697354031) / Math.Sqrt((4790.885284200757 / (44)) + (8443.776091204752 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.086325467351079 = (4316.640673801445 - 4689.276697354031) / 4316.640673801445 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.956085074123987 > 20.666757632171933. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.125920884185668 (T) = (0 -20833.601517925774) / Math.Sqrt((36816.08531710939 / (44)) + (22736.953681611212 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05951027342382724 = (19663.42567930144 - 20833.601517925774) / 19663.42567930144 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.9340185055048 > 10.700474496522272. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.77812116635703 (T) = (0 -10864.81550048756) / Math.Sqrt((23838.313740682603 / (44)) + (10429.242020829468 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07584812892543703 = (10098.837566729235 - 10864.81550048756) / 10098.837566729235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<SimpleStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.225941553030303 > 19.442801105140006. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.179585928704878 (T) = (0 -19684.771569527496) / Math.Sqrt((46343.378922346 / (44)) + (53823.54751268813 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05931884272259656 = (18582.48034079607 - 19684.771569527496) / 18582.48034079607 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Hypot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.Hypot.html>) 811.02 μs 1.08 ms 1.33 0.18 False
[SinCosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.SinCosPi.html>) 1.40 ms 1.58 ms 1.13 0.10 False
[Max - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.Max.html>) 61.92 μs 129.18 μs 2.09 0.06 False
[CopySign - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.CopySign.html>) 219.40 μs 277.65 μs 1.27 0.05 False
[ILogB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.ILogB.html>) 249.23 μs 269.07 μs 1.08 0.04 False
[SinPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.SinPi.html>) 686.65 μs 779.76 μs 1.14 0.11 False
[Round - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.Round.html>) 317.19 μs 511.42 μs 1.61 0.09 False
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.Min.html>) 62.16 μs 129.14 μs 2.08 0.08 False
[TanPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.TanPi.html>) 827.42 μs 950.64 μs 1.15 0.12 False
[Abs - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.Abs.html>) 56.65 μs 112.70 μs 1.99 0.06 False
[CosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Double.CosPi.html>) 726.64 μs 784.35 μs 1.08 0.08 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Double.Hypot ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0797428571428571 > 852.0574137952302. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.84741683089046 (T) = (0 -1097165.792857804) / Math.Sqrt((854242766.0858712 / (43)) + (1027872940.4694506 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.3283641903837274 = (825952.5518682218 - 1097165.792857804) / 825952.5518682218 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.SinCosPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5801395837499999 > 1.4779086402947446. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/15/2022 5:29:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.00898684766763 (T) = (0 -1588064.6765813488) / Math.Sqrt((339102541.03009367 / (44)) + (156253345.0646596 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12351316104607647 = (1413481.1514826755 - 1588064.6765813488) / 1413481.1514826755 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.Max ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.18495720756516 > 65.0213190402668. IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -117.6382133429767 (T) = (0 -129385.01408153583) / Math.Sqrt((11764504.102734784 / (44)) + (641280.3995934491 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0256477968561517 = (63873.40103365655 - 129385.01408153583) / 63873.40103365655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.CopySign ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 277.6539460526315 > 232.8892933323378. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.93255073477788 (T) = (0 -278262.625095051) / Math.Sqrt((25493541.381847754 / (44)) + (2051932.7292813887 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2460747665728321 = (223311.33938325103 - 278262.625095051) / 223311.33938325103 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.ILogB ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 269.071506080665 > 262.5518639201709. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.39419417640053 (T) = (0 -269538.84733087354) / Math.Sqrt((4189935.037293694 / (43)) + (889641.4526222461 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.07418491425045083 = (250924.06694144785 - 269538.84733087354) / 250924.06694144785 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.SinPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 779.7571430803572 > 739.9317196732953. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.737899070301733 (T) = (0 -791736.9350777228) / Math.Sqrt((164967219.30311003 / (44)) + (284265191.80314386 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.13131570191684422 = (699837.3077791139 - 791736.9350777228) / 699837.3077791139 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.Round ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 511.4200751728111 > 333.026970321588. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -122.40714934133665 (T) = (0 -511050.84998024977) / Math.Sqrt((18707429.18388482 / (44)) + (30945782.366609786 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.607256867608188 = (317964.6391810174 - 511050.84998024977) / 317964.6391810174 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.Min ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.13936019345238 > 65.26599840101305. IsChangePoint: Marked as a change because one of 11/15/2022 5:29:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -208.52686535872664 (T) = (0 -129773.2381261946) / Math.Sqrt((52292.040148360924 / (44)) + (1556144.6074918956 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0855523058582204 = (62224.878158973785 - 129773.2381261946) / 62224.878158973785 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.TanPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 950.644694327731 > 871.234169847862. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.54790540278502 (T) = (0 -963083.4668655536) / Math.Sqrt((320397387.95136493 / (44)) + (278454499.23316824 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14887227503552206 = (838285.9329038783 - 963083.4668655536) / 838285.9329038783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.Abs ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.70413660071944 > 59.4694776762385. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -731.505936143309 (T) = (0 -112703.64793212134) / Math.Sqrt((253928.78971331913 / (44)) + (1437.9711763496914 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.9886623698708049 = (56673.09325083836 - 112703.64793212134) / 56673.09325083836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Double.CosPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 784.34754375 > 738.6446882420569. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.597900558833974 (T) = (0 -775689.6516716741) / Math.Sqrt((366274850.377402 / (44)) + (96509229.60655145 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11977729405468221 = (692717.7893230212 - 775689.6516716741) / 692717.7893230212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).LessThanOrEqualAllBenchmark.html>) 76.44 ns 345.01 ns 4.51 0.05 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).LessThanAnyBenchmark.html>) 207.49 ns 297.74 ns 1.43 0.21 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).GreaterThanOrEqualAnyBenchmark.html>) 213.68 ns 301.13 ns 1.41 0.20 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).GetHashCodeBenchmark.html>) 791.39 ns 884.93 ns 1.12 0.04 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).AllBitsSetBenchmark.html>) 43.30 ns 88.96 ns 2.05 0.04 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).EqualsAnyBenchmark.html>) 222.48 ns 294.32 ns 1.32 0.26 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).ConditionalSelectBenchmark.html>) 181.47 ns 427.45 ns 2.36 0.06 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).LessThanOrEqualAnyBenchmark.html>) 227.70 ns 300.93 ns 1.32 0.16 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).OnesComplementBenchmark.html>) 102.31 ns 162.39 ns 1.59 0.10 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).GreaterThanAnyBenchmark.html>) 210.00 ns 307.12 ns 1.46 0.20 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).AndNotBenchmark.html>) 132.86 ns 268.09 ns 2.02 0.06 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).OnesComplementOperatorBenchmark.html>) 109.46 ns 154.28 ns 1.41 0.11 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).GreaterThanAllBenchmark.html>) 74.11 ns 339.06 ns 4.58 0.07 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).GreaterThanOrEqualAllBenchmark.html>) 75.75 ns 353.04 ns 4.66 0.07 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).LessThanAllBenchmark.html>) 74.80 ns 341.81 ns 4.57 0.06 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Single).SumBenchmark.html>) 72.08 ns 86.03 ns 1.19 0.10 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 345.0110963655312 > 79.86145437053322. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -239.2285034129826 (T) = (0 -348.79336828000737) / Math.Sqrt((11.478058260214347 / (44)) + (15.45824764243477 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.533725346143167 = (76.93306092675537 - 348.79336828000737) / 76.93306092675537 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 297.7423156204261 > 219.17797945372024. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -77.45896833214903 (T) = (0 -296.9413595269366) / Math.Sqrt((34.10217198797781 / (44)) + (6.43897164475108 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.40822858861333305 = (210.8616185809233 - 296.9413595269366) / 210.8616185809233 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 301.12902627130313 > 231.9868669864484. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.6501765545847 (T) = (0 -303.05926182791006) / Math.Sqrt((58.90170748698687 / (44)) + (14.834884173517763 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3891892874415143 = (218.1554843300424 - 303.05926182791006) / 218.1554843300424 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 884.9272452785073 > 815.3338406006375. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.0593176456451 (T) = (0 -890.8224989678762) / Math.Sqrt((177.39641136946258 / (44)) + (54.925586317095174 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14250527357348133 = (779.7097480185785 - 890.8224989678762) / 779.7097480185785 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.95659044408667 > 46.30864822905507. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.626614029964354 (T) = (0 -88.25917651804743) / Math.Sqrt((919.7884289556395 / (44)) + (0.7620310641847813 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4120930336857577 = (62.50238080112809 - 88.25917651804743) / 62.50238080112809 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 294.31839802302073 > 224.66339686170951. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.38476220145084 (T) = (0 -299.78228068577675) / Math.Sqrt((77.79502729780798 / (43)) + (77.70680521619903 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.4145572766233274 = (211.92657635001066 - 299.78228068577675) / 211.92657635001066 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 427.45422865311616 > 190.51706687352902. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.02081241012304 (T) = (0 -422.04822139258636) / Math.Sqrt((922.6530318181102 / (44)) + (22.286141888517374 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1183817199009867 = (199.23143096812265 - 422.04822139258636) / 199.23143096812265 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 300.93411172601117 > 226.89551924189135. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -71.00683411887503 (T) = (0 -302.3907502441969) / Math.Sqrt((40.789650036646215 / (44)) + (8.012696362585647 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.39635569719916763 = (216.5571070829138 - 302.3907502441969) / 216.5571070829138 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 162.38824918917624 > 110.1409801846061. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.941460107631043 (T) = (0 -154.69396149695032) / Math.Sqrt((944.6966359828481 / (44)) + (11.68264581121286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2684851459333678 = (121.95173273639283 - 154.69396149695032) / 121.95173273639283 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 307.1195488273198 > 221.7584443586415. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.3115989564729 (T) = (0 -298.453851464811) / Math.Sqrt((37.15103263954859 / (44)) + (16.829904114520982 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4139589713249335 = (211.07674092208515 - 298.453851464811) / 211.07674092208515 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 268.0945044480033 > 139.7860415931581. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.80837240064469 (T) = (0 -266.500627966274) / Math.Sqrt((896.4269065621027 / (44)) + (13.968786476404931 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7533757424826708 = (151.99287951190982 - 266.500627966274) / 151.99287951190982 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.28084704929915 > 114.0903517146588. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.988828584683633 (T) = (0 -155.55200657523432) / Math.Sqrt((921.9064765720254 / (44)) + (27.06176410671634 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2727981318572912 = (122.2126295457787 - 155.55200657523432) / 122.2126295457787 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 339.0572853095897 > 77.96901925002948. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -134.611359805044 (T) = (0 -341.42355440499074) / Math.Sqrt((1.6170856914412086 / (43)) + (58.26280412152288 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -3.56169933698401 = (74.8456943746592 - 341.42355440499074) / 74.8456943746592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 353.039704474349 > 79.49733193753863. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -149.8504107580898 (T) = (0 -349.5532629782738) / Math.Sqrt((0.9936278897054754 / (44)) + (49.548303297365166 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.5828436694114187 = (76.27431529279448 - 349.5532629782738) / 76.27431529279448 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 341.81260552396515 > 78.32862128848537. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -200.15673061380602 (T) = (0 -341.8496962713413) / Math.Sqrt((1.367548173408882 / (44)) + (26.214042060480285 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.563710574702508 = (74.90608588683897 - 341.8496962713413) / 74.90608588683897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 86.03186997968382 > 75.6775442722799. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.51515301259063 (T) = (0 -87.18419425460911) / Math.Sqrt((4.739533493381855 / (44)) + (4.897771498626322 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20507545462478527 = (72.34749817533621 - 87.18419425460911) / 72.34749817533621 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20OneEscaped).html>) 451.74 ms 518.14 ms 1.15 0.04 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20NoneEscaped).html>) 374.44 ms 404.38 ms 1.08 0.05 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>) 875.28 ms 1.10 secs 1.26 0.09 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20NoneEscaped).html>) 768.41 ms 1.02 secs 1.33 0.09 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20OneEscaped).html>) 418.62 ms 489.98 ms 1.17 0.02 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20OneEscaped).html>) 703.92 ms 908.63 ms 1.29 0.07 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>) 1.32 secs 1.43 secs 1.08 0.07 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20OneEscaped).html>) 721.44 ms 935.17 ms 1.30 0.06 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20OneEscaped).html>) 689.55 ms 908.03 ms 1.32 0.07 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20NoneEscaped).html>) 764.79 ms 1.02 secs 1.33 0.11 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>) 892.76 ms 1.12 secs 1.26 0.08 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>) 896.35 ms 1.13 secs 1.26 0.08 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>) 1.33 secs 1.42 secs 1.07 0.09 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20NoneEscaped).html>) 780.47 ms 1.03 secs 1.32 0.11 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20NoneEscaped).html>) 382.95 ms 422.83 ms 1.10 0.04 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20NoneEscaped).html>) 361.87 ms 406.30 ms 1.12 0.04 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20NoneEscaped).html>) 383.46 ms 426.41 ms 1.11 0.04 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20OneEscaped).html>) 424.79 ms 495.98 ms 1.17 0.03 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20OneEscaped).html>) 698.56 ms 926.79 ms 1.33 0.06 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>) 874.74 ms 1.09 secs 1.25 0.06 False
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20NoneEscaped).html>) 782.68 ms 1.05 secs 1.34 0.08 False
[WriteStringsUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20OneEscaped).html>) 445.48 ms 521.58 ms 1.17 0.02 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 518.1397141428571 > 476.61750479. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.92179827022149 (T) = (0 -525773420.5880097) / Math.Sqrt((33511825576050.598 / (44)) + (68966243461022.305 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16953323305612994 = (449558341.5052696 - 525773420.5880097) / 449558341.5052696 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 404.3839333333333 > 383.967542425. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.00271374615024 (T) = (0 -408031429.4214652) / Math.Sqrt((15794425986815.127 / (44)) + (19627745975141.438 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11662563696631768 = (365414706.5170538 - 408031429.4214652) / 365414706.5170538 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1017145712857144 > 924.2628751125001. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.25187042913615 (T) = (0 -1102547555.8552992) / Math.Sqrt((461872971526345.5 / (44)) + (87367996597285.9 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23680287775623635 = (891449701.2292708 - 1102547555.8552992) / 891449701.2292708 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.020204846 > 864.09315986. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 7:24:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.131154194264312 (T) = (0 -1021871614.8016117) / Math.Sqrt((6084286174807716 / (43)) + (525254096447254.44 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2449059534751117 = (820842419.4205936 - 1021871614.8016117) / 820842419.4205936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 489.97913346666667 > 442.645607425. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -58.093495811058126 (T) = (0 -491195317.3530611) / Math.Sqrt((20157300801643.36 / (44)) + (13485975949237.197 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.16415008227017056 = (421934701.40482 - 491195317.3530611) / 421934701.40482 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 908.6329998 > 727.5482324024999. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.319979112765854 (T) = (0 -911645613.1442735) / Math.Sqrt((3003996941409677 / (44)) + (68640174715717.484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2793118366917323 = (712606252.0470113 - 911645613.1442735) / 712606252.0470113 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4261729333333333 > 1.384395914755. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.350220269701783 (T) = (0 -1409790289.1302075) / Math.Sqrt((5585569998272778 / (44)) + (310802019941974 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05790549757266911 = (1332624031.5084164 - 1409790289.1302075) / 1332624031.5084164 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 935.1698666666666 > 748.0068848875001. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.491304783220194 (T) = (0 -931982275.2371674) / Math.Sqrt((2447092520957474 / (44)) + (35308507468166.65 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2761927462810534 = (730283319.6263276 - 931982275.2371674) / 730283319.6263276 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 908.0294616153845 > 730.908119925. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.01953133680684 (T) = (0 -914350692.4432758) / Math.Sqrt((2402575187953812 / (43)) + (46302053635555.71 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.2802726917507146 = (714184328.334726 - 914350692.4432758) / 714184328.334726 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0166896153076923 > 805.8886149650001. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.715927449661997 (T) = (0 -1015025704.0800731) / Math.Sqrt((3657456980115262 / (44)) + (48500548922487.99 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2625803464480115 = (803929593.0240177 - 1015025704.0800731) / 803929593.0240177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1238326153076923 > 939.3682125000001. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.61347170251855 (T) = (0 -1121838546.9806104) / Math.Sqrt((301944563220797.2 / (44)) + (54540698804981.5 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24267321862392968 = (902762311.25579 - 1121838546.9806104) / 902762311.25579 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1277012147142855 > 949.685449965. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.32040763549419 (T) = (0 -1133109378.2062027) / Math.Sqrt((292606104307710.2 / (44)) + (81181870575250.3 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24555886389972703 = (909719653.6007495 - 1133109378.2062027) / 909719653.6007495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4196345334666667 > 1.4065316347475. IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/22/2022 11:54:51 PM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.71838512290591 (T) = (0 -1440772039.5360196) / Math.Sqrt((474638423368247.8 / (43)) + (600234272183730.4 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.06732015561133319 = (1349896778.3577392 - 1440772039.5360196) / 1349896778.3577392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0284844288571429 > 820.08143008. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.07356183788616 (T) = (0 -1050336567.9439079) / Math.Sqrt((3792319855907764 / (44)) + (1325035191637206.5 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27347257189861807 = (824781460.6465869 - 1050336567.9439079) / 824781460.6465869 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 422.82859973333336 > 397.57023765. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.634661153209436 (T) = (0 -423168903.6835165) / Math.Sqrt((18965803125063.504 / (44)) + (13589890555922.365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11802865866315494 = (378495578.2703338 - 423168903.6835165) / 378495578.2703338 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 406.2975000714286 > 380.681748553077. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.180515005782908 (T) = (0 -408032690.8582418) / Math.Sqrt((86690906549041.23 / (44)) + (17773729164631.668 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12311303172338195 = (363305098.7148892 - 408032690.8582418) / 363305098.7148892 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 426.40542835714285 > 400.09031996500005. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.93971991981528 (T) = (0 -429827225.0307936) / Math.Sqrt((10267188809420.6 / (44)) + (100583225616921.02 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13131293154915666 = (379936632.07067937 - 429827225.0307936) / 379936632.07067937 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 495.97671428571425 > 448.093049925. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.277446590165304 (T) = (0 -497342829.65404147) / Math.Sqrt((25180303498977.887 / (44)) + (42324112096631.2 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17026266697758252 = (424983931.9735973 - 497342829.65404147) / 424983931.9735973 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 926.7877142142858 > 737.0836674288461. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.8152294486496 (T) = (0 -937161989.3538094) / Math.Sqrt((2255058289180313.5 / (44)) + (63528264531951.46 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2838392592240815 = (729968321.6730771 - 937161989.3538094) / 729968321.6730771 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0924380666 > 916.152194755. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -59.93694735422641 (T) = (0 -1099824402.0339196) / Math.Sqrt((247923019392704.88 / (44)) + (109034871448949.1 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24341191436009324 = (884521363.622232 - 1099824402.0339196) / 884521363.622232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: NoneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0526105717142857 > 823.1856976400001. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.761204138106358 (T) = (0 -1037295011.777973) / Math.Sqrt((4715137464975673 / (44)) + (61810711835938.336 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25151322994989733 = (828832637.9254495 - 1037295011.777973) / 828832637.9254495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: OneEscaped) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 521.5819332 > 464.76405015. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.691905866496413 (T) = (0 -519045532.63777775) / Math.Sqrt((304014160179385.4 / (44)) + (23156142307605.22 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16051230664707755 = (447255517.8129828 - 519045532.63777775) / 447255517.8129828 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.BenchF.Regula

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.BenchF.Regula.Test.html>) 2.86 secs 4.47 secs 1.57 0.08 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Regula*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Regula.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.472096 > 3.0219357003150003. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.98429857427287 (T) = (0 -3736715665.1030893) / Math.Sqrt((2329540340950145 / (44)) + (44650230252396050 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2832351344215799 = (2911949310.6673856 - 3736715665.1030893) / 2911949310.6673856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DistanceSquaredBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark.html>) 56.65 ns 69.69 ns 1.23 0.14 False
[TransformNormalByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark.html>) 105.82 ns 124.14 ns 1.17 0.10 False
[UnitZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.UnitZBenchmark.html>) 44.71 ns 57.28 ns 1.28 0.14 False
[MultiplyFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark.html>) 46.32 ns 59.86 ns 1.29 0.14 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark.html>) 46.44 ns 60.13 ns 1.29 0.15 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.NegateBenchmark.html>) 45.09 ns 58.43 ns 1.30 0.19 False
[DivideByVector3OperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark.html>) 46.66 ns 59.76 ns 1.28 0.21 False
[TransformByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark.html>) 201.07 ns 219.83 ns 1.09 0.15 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MaxBenchmark.html>) 62.46 ns 82.82 ns 1.33 0.10 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MinBenchmark.html>) 63.10 ns 80.61 ns 1.28 0.11 False
[NegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark.html>) 47.53 ns 59.26 ns 1.25 0.16 False
[UnitYBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.UnitYBenchmark.html>) 43.24 ns 58.71 ns 1.36 0.15 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.OneBenchmark.html>) 19.88 ns 76.97 ns 3.87 0.24 False
[CreateFromScalar - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.CreateFromScalar.html>) 19.61 ns 55.96 ns 2.85 0.26 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark.html>) 46.38 ns 61.76 ns 1.33 0.14 False
[DivideByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark.html>) 54.98 ns 102.40 ns 1.86 0.11 False
[ReflectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.ReflectBenchmark.html>) 105.68 ns 169.42 ns 1.60 0.13 False
[DivideByVector3Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark.html>) 44.54 ns 63.16 ns 1.42 0.16 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark.html>) 36.00 ns 48.29 ns 1.34 0.18 False
[TransformByMatrix4x4Benchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark.html>) 115.10 ns 131.03 ns 1.14 0.12 False
[SubtractFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark.html>) 46.83 ns 61.09 ns 1.30 0.11 False
[CreateFromVector2WithScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark.html>) 28.05 ns 61.06 ns 2.18 0.21 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.AbsBenchmark.html>) 35.88 ns 98.77 ns 2.75 0.13 False
[AddFunctionBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark.html>) 46.21 ns 60.43 ns 1.31 0.15 False
[CreateFromScalarXYZBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark.html>) 22.88 ns 38.25 ns 1.67 0.24 False
[LerpBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.LerpBenchmark.html>) 145.66 ns 263.29 ns 1.81 0.11 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark.html>) 71.75 ns 102.08 ns 1.42 0.13 False
[SubtractOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark.html>) 49.98 ns 60.38 ns 1.21 0.15 False
[DistanceBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DistanceBenchmark.html>) 58.34 ns 71.89 ns 1.23 0.13 False
[DivideByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark.html>) 52.83 ns 104.69 ns 1.98 0.14 False
[NormalizeBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark.html>) 79.21 ns 120.45 ns 1.52 0.11 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark.html>) 68.30 ns 103.94 ns 1.52 0.16 False
[UnitXBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.UnitXBenchmark.html>) 45.32 ns 59.04 ns 1.30 0.11 False
[CrossBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Vector3.CrossBenchmark.html>) 66.79 ns 81.58 ns 1.22 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Vector3.DistanceSquaredBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.6910503292956 > 59.872953378397014. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.14334899715622 (T) = (0 -71.7256050476792) / Math.Sqrt((2.3921617227715735 / (44)) + (1.962467206260169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25763524939206384 = (57.032120467640425 - 71.7256050476792) / 57.032120467640425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.TransformNormalByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 124.13911882068626 > 111.92864390671758. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/14/2022 9:30:20 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.9185105367862 (T) = (0 -122.93866621130921) / Math.Sqrt((3.1140225054283452 / (44)) + (3.171735716475723 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14280943234752075 = (107.57582387010294 - 122.93866621130921) / 107.57582387010294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.UnitZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.279625416212994 > 45.803718669755106. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.19074047530175 (T) = (0 -58.90135186207126) / Math.Sqrt((0.905705949671245 / (43)) + (2.02933798563345 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.3570242478894625 = (43.40478952655319 - 58.90135186207126) / 43.40478952655319 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MultiplyFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.85889616886244 > 48.72248983301346. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.009434619006367 (T) = (0 -62.238573694931304) / Math.Sqrt((0.6482282331876037 / (44)) + (8.251029717510912 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3399248822094185 = (46.449300644604314 - 62.238573694931304) / 46.449300644604314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.132587963392176 > 48.447991739835636. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.985464535424196 (T) = (0 -61.596536313083114) / Math.Sqrt((9.312420474907865 / (44)) + (4.356803469937244 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29854504077899763 = (47.43504027871943 - 61.596536313083114) / 47.43504027871943 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.432566048867635 > 47.78681610432081. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.85673278451828 (T) = (0 -61.16893849154963) / Math.Sqrt((18.94290827014465 / (43)) + (3.2754726565959267 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.3105519472630484 = (46.674180767343564 - 61.16893849154963) / 46.674180767343564 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.DivideByVector3OperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.75718346557389 > 49.0025058305935. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.140245350000974 (T) = (0 -61.56396727739806) / Math.Sqrt((3.826635676119893 / (44)) + (6.04320265884416 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.31636740568044125 = (46.76807326870505 - 61.56396727739806) / 46.76807326870505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.TransformByQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 219.827020028509 > 212.13797388514115. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/17/2022 7:01:39 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -8.570742962769987 (T) = (0 -220.82320790200967) / Math.Sqrt((26.061612929047893 / (44)) + (53.06659315394938 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08563192631569204 = (203.40522653144254 - 220.82320790200967) / 203.40522653144254 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.81788913640223 > 66.00486745735388. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.212994638599186 (T) = (0 -81.0992267438716) / Math.Sqrt((2.631202776668451 / (44)) + (27.30379190994239 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2876525832982785 = (62.98222656932717 - 81.0992267438716) / 62.98222656932717 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.61466282032282 > 66.46922959742847. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.67045876162263 (T) = (0 -81.70669513744451) / Math.Sqrt((3.3120792177597957 / (44)) + (2.9348660201432133 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2834791513366907 = (63.66032128558563 - 81.70669513744451) / 63.66032128558563 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.NegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.25650245574084 > 49.17026923743281. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.766908781158637 (T) = (0 -59.7531144308499) / Math.Sqrt((17.248163629612797 / (44)) + (0.3863368586552475 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2897126346083003 = (46.33056452067523 - 59.7531144308499) / 46.33056452067523 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.UnitYBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.70899724139268 > 46.17766809865701. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.0075846877719 (T) = (0 -59.348324610823546) / Math.Sqrt((1.0855791085252835 / (43)) + (1.645498376835075 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.36277872492913815 = (43.549494518201804 - 59.348324610823546) / 43.549494518201804 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.96677716673015 > 21.018901448183556. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -186.10386031859602 (T) = (0 -74.8490703315138) / Math.Sqrt((1.1170918799359526 / (44)) + (0.9069701305984875 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.6836389626971697 = (20.319328546983638 - 74.8490703315138) / 20.319328546983638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.CreateFromScalar ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.95512323020694 > 21.36281932616838. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.08820856705125 (T) = (0 -54.250113889346835) / Math.Sqrt((0.637566371668023 / (44)) + (32.29977740386565 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.678193200009269 = (20.256236140529026 - 54.250113889346835) / 20.256236140529026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.76037607445108 > 48.0140066399034. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.993277269443606 (T) = (0 -60.21544958542847) / Math.Sqrt((3.907010954230937 / (44)) + (1.744941788302542 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2860817574166651 = (46.820856635415176 - 60.21544958542847) / 46.820856635415176 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.DivideByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.39529982462355 > 56.30764559399837. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.933549276663186 (T) = (0 -105.02459823804412) / Math.Sqrt((18.87555735542118 / (44)) + (25.045207611508097 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.9299924270630188 = (54.41710380069529 - 105.02459823804412) / 54.41710380069529 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.ReflectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.41968146152823 > 112.10369355453221. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -56.47128293093286 (T) = (0 -172.4687656013136) / Math.Sqrt((17.182343081969865 / (44)) + (13.63670517859633 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5955890096480997 = (108.09097114510136 - 172.4687656013136) / 108.09097114510136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.DivideByVector3Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.159345354425355 > 48.494914869407744. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.50368856101929 (T) = (0 -60.228050152001764) / Math.Sqrt((1.2761935729179266 / (44)) + (1.3855493297698498 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2865442751184166 = (46.81381847232442 - 60.228050152001764) / 46.81381847232442 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.290923061979164 > 36.85848352595452. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.511481851318898 (T) = (0 -49.0776848005262) / Math.Sqrt((6.785457357560412 / (44)) + (1.4903099664277892 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3932982747581951 = (35.22410505320087 - 49.0776848005262) / 35.22410505320087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.TransformByMatrix4x4Benchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 131.02596195974107 > 122.2443581599268. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.321232172440853 (T) = (0 -133.05204898500537) / Math.Sqrt((64.87677479096544 / (44)) + (9.164095348960581 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11255626117569006 = (119.59129944979418 - 133.05204898500537) / 119.59129944979418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.SubtractFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.08585170589539 > 49.752606421762195. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.389702150901467 (T) = (0 -60.59694726591466) / Math.Sqrt((11.530990964582918 / (44)) + (1.3005009056041081 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27910868976493547 = (47.374353525071186 - 60.59694726591466) / 47.374353525071186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.CreateFromVector2WithScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.05969541688617 > 29.987417631573404. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -68.12691463349148 (T) = (0 -61.39338686234257) / Math.Sqrt((4.938173483547554 / (44)) + (1.8015303739311916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1499625554918198 = (28.555560981990393 - 61.39338686234257) / 28.555560981990393 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 98.76877134058395 > 37.74475257192357. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -107.84612002519916 (T) = (0 -100.34268418669738) / Math.Sqrt((0.6387514028637471 / (44)) + (5.165382082315333 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.8078787246140249 = (35.73611755632026 - 100.34268418669738) / 35.73611755632026 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.AddFunctionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.43021819778376 > 48.481525513209. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.85135084592921 (T) = (0 -60.977405508778325) / Math.Sqrt((4.380764402161667 / (44)) + (0.9230003018580426 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3143708246282394 = (46.39284771557932 - 60.977405508778325) / 46.39284771557932 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.CreateFromScalarXYZBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 38.249681518000614 > 24.427226180555408. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.07795975073523 (T) = (0 -38.134733986902965) / Math.Sqrt((0.7154940926242958 / (44)) + (2.344492015288212 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6474455770245734 = (23.147795908243314 - 38.134733986902965) / 23.147795908243314 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.LerpBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 263.28747409154005 > 155.09889431742684. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -76.67794801508964 (T) = (0 -262.1016873184092) / Math.Sqrt((11.973438326711268 / (44)) + (29.58025158279977 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.780163304258753 = (147.2346310540012 - 262.1016873184092) / 147.2346310540012 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 102.07523651848831 > 55.68823406560502. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.04926274383385 (T) = (0 -102.66059902391474) / Math.Sqrt((9.474855455458052 / (44)) + (2.433070072244916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8979870774988491 = (54.08919809886166 - 102.66059902391474) / 54.08919809886166 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.SubtractOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 60.37799517981826 > 48.4220339242217. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.99423134780405 (T) = (0 -60.90032761856441) / Math.Sqrt((1.1288706900021717 / (44)) + (3.297712407122481 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.308722714409725 = (46.53417179057091 - 60.90032761856441) / 46.53417179057091 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.DistanceBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 71.88574994755257 > 61.98560971341081. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.58650996868425 (T) = (0 -74.02864300503202) / Math.Sqrt((1.3839447913259142 / (44)) + (8.081903637029043 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24968272070723493 = (59.237950384027776 - 74.02864300503202) / 59.237950384027776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.DivideByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.68534185986672 > 55.80470434353156. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -112.53743015813531 (T) = (0 -102.1366902925834) / Math.Sqrt((3.647143720684353 / (44)) + (1.5447805124691034 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.9048892617972086 = (53.61817736125004 - 102.1366902925834) / 53.61817736125004 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.NormalizeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 120.45335516994417 > 76.04934562395316. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.803435301809387 (T) = (0 -124.5444790293511) / Math.Sqrt((29.605314400068067 / (43)) + (35.104048058956295 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.6736057756616808 = (74.41685541513556 - 124.5444790293511) / 74.41685541513556 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 103.9435847846202 > 56.68618156894957. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.20203104147607 (T) = (0 -104.46102322901048) / Math.Sqrt((6.641041005849388 / (43)) + (30.420611791128735 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.9413220045021992 = (53.80922020496891 - 104.46102322901048) / 53.80922020496891 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.UnitXBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 59.04217865263297 > 45.562208739279896. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.65331482354735 (T) = (0 -59.13563562174108) / Math.Sqrt((0.6864340813202037 / (43)) + (0.9294986845878881 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.3520327159465332 = (43.7383170719662 - 59.13563562174108) / 43.7383170719662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Vector3.CrossBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 81.57655608233487 > 70.08833347665082. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.94439968143257 (T) = (0 -81.95569572255862) / Math.Sqrt((15.780364340035703 / (44)) + (24.652691873570756 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20727950901844508 = (67.88460759115434 - 81.95569572255862) / 67.88460759115434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20Compiled).html>) 162.60 μs 196.52 μs 1.21 0.01 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20None).html>) 8.80 μs 9.53 μs 1.08 0.04 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options%3a%20IgnoreCase%2c%20Compiled).html>) 283.01 μs 314.05 μs 1.11 0.00 False
[ReplaceWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options%3a%20None).html>) 1.43 ms 1.55 ms 1.09 0.01 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20Compiled).html>) 9.77 μs 10.60 μs 1.08 0.01 False
[MatchesWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options%3a%20Compiled).html>) 147.83 μs 175.25 μs 1.19 0.04 False
[CtorInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options%3a%20None).html>) 181.58 μs 196.12 μs 1.08 0.00 False
[MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20Compiled).html>) 6.84 μs 8.35 μs 1.22 0.03 False
[Date_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 11.12 μs 11.68 μs 1.05 0.01 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20None).html>) 5.17 μs 5.82 μs 1.13 0.03 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20None).html>) 12.34 μs 13.25 μs 1.07 0.03 False
[CtorInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options%3a%20Compiled).html>) 181.35 μs 199.76 μs 1.10 0.00 False
[Email_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options%3a%20Compiled).html>) 12.32 μs 13.13 μs 1.07 0.02 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 5.34 μs 5.80 μs 1.09 0.02 False
[MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20None).html>) 39.83 μs 43.46 μs 1.09 0.02 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options%3a%20None).html>) 177.47 μs 192.91 μs 1.09 0.01 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 9.90 μs 10.77 μs 1.09 0.01 False
[MatchWord - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options%3a%20IgnoreCase%2c%20Compiled).html>) 8.20 μs 10.33 μs 1.26 0.03 False
[Backtracking - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options%3a%20None).html>) 38.97 μs 43.37 μs 1.11 0.03 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20IgnoreCase%2c%20Compiled).html>) 8.57 μs 9.38 μs 1.09 0.01 False
[SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20Compiled).html>) 151.83 μs 185.00 μs 1.22 0.02 False
[Uri_IsNotMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options%3a%20Compiled).html>) 8.69 μs 9.37 μs 1.08 0.01 False
[Ctor - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options%3a%20Compiled).html>) 182.01 μs 191.97 μs 1.05 0.00 False
[CtorInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options%3a%20IgnoreCase%2c%20Compiled).html>) 282.71 μs 319.79 μs 1.13 0.00 False
[Date_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options%3a%20Compiled).html>) 5.19 μs 5.61 μs 1.08 0.02 False
[Uri_IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options%3a%20None).html>) 9.79 μs 10.63 μs 1.09 0.03 False
[SplitWords - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options%3a%20None).html>) 1.41 ms 1.56 ms 1.11 0.01 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 196.52198683035715 > 174.71889412954343. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.46107147966217 (T) = (0 -197396.81263696044) / Math.Sqrt((5947879.110253171 / (44)) + (5054950.66835612 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20798775691898708 = (163409.61363749875 - 197396.81263696044) / 163409.61363749875 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.529965049291437 > 9.223620570967313. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.90375676872227 (T) = (0 -9623.874568943855) / Math.Sqrt((15441.507473168169 / (44)) + (28220.549314349882 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10829766528192482 = (8683.47454877636 - 9623.874568943855) / 8683.47454877636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.0525948076924 > 297.08267370535714. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.669029940672903 (T) = (0 -317385.5594933013) / Math.Sqrt((22550225.451711178 / (44)) + (17710844.2986068 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10746883477852465 = (286586.4478766783 - 317385.5594933013) / 286586.4478766783 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5511156241071427 > 1.5174710813068184. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.74102869505873 (T) = (0 -1568172.7587073413) / Math.Sqrt((568937269.3657097 / (44)) + (592704586.8887191 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08378774590737594 = (1446937.1559413834 - 1568172.7587073413) / 1446937.1559413834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.595298312272314 > 10.30343883645393. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.278829777725477 (T) = (0 -10746.400659553068) / Math.Sqrt((13830.560057215831 / (43)) + (17802.262181593902 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.08756953659859279 = (9881.115917574123 - 10746.400659553068) / 9881.115917574123 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 175.2500497519841 > 155.67391315860849. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.703582262320296 (T) = (0 -178466.93100115153) / Math.Sqrt((14644478.476312565 / (44)) + (5556516.500855732 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19484387006047157 = (149364.22696977074 - 178466.93100115153) / 149364.22696977074 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 196.12399626116076 > 190.97037788154068. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 10:22:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.814853650084494 (T) = (0 -196742.4552829943) / Math.Sqrt((4154473.146795419 / (43)) + (2067149.7301977759 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.07512268070995758 = (182995.35375169964 - 196742.4552829943) / 182995.35375169964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.352113681491465 > 7.199135756007972. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.99330579495264 (T) = (0 -8245.097891630114) / Math.Sqrt((10817.137445298871 / (44)) + (5386.109159615217 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19623623869440676 = (6892.533117562931 - 8245.097891630114) / 6892.533117562931 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.677299315252418 > 11.63399329558646. IsChangePoint: Marked as a change because one of 10/21/2022 10:18:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.29714431088082 (T) = (0 -11745.639998688186) / Math.Sqrt((16266.072350544728 / (44)) + (7209.325055730534 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05859917625717932 = (11095.45544916867 - 11745.639998688186) / 11095.45544916867 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.821636654903419 > 5.523046509559906. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 10/22/2022 10:22:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.322727527036918 (T) = (0 -5686.840306467574) / Math.Sqrt((7443.186642114852 / (44)) + (5257.854418546859 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07462292215325646 = (5291.9402603777235 - 5686.840306467574) / 5291.9402603777235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.251989517841974 > 12.966715495969142. IsChangePoint: Marked as a change because one of 10/29/2022 4:38:27 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.911770644397306 (T) = (0 -13149.08663289538) / Math.Sqrt((20065.324480957897 / (44)) + (34578.76530389349 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06336839208852503 = (12365.504495643052 - 13149.08663289538) / 12365.504495643052 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 199.7622887394515 > 191.41732851981777. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.386991867364763 (T) = (0 -197064.2371789821) / Math.Sqrt((5434213.342559655 / (43)) + (3688368.6497063367 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.07447532335868197 = (183405.08422565047 - 197064.2371789821) / 183405.08422565047 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.134743212986288 > 12.978376390704332. IsChangePoint: Marked as a change because one of 10/23/2022 1:14:54 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.193945932957917 (T) = (0 -13105.296403106422) / Math.Sqrt((70840.33347928889 / (44)) + (31688.06717190627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.054979445615974884 = (12422.323920685092 - 13105.296403106422) / 12422.323920685092 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.801674883898886 > 5.6047077636508105. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.807642070196934 (T) = (0 -5701.836548324616) / Math.Sqrt((5779.894020029335 / (44)) + (4576.000302915369 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07825182567496543 = (5288.037926349323 - 5701.836548324616) / 5288.037926349323 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.46483148661126 > 42.04068170329649. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.123843196788805 (T) = (0 -43600.50605879245) / Math.Sqrt((325073.2204799397 / (44)) + (179353.5450717338 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0796399585252088 = (40384.301928163964 - 43600.50605879245) / 40384.301928163964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 192.90669020061728 > 188.10245574855506. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.248009006089056 (T) = (0 -192821.92897018619) / Math.Sqrt((5313149.461952379 / (44)) + (709640.5992087844 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.073608539229524 = (179601.70949139897 - 192821.92897018619) / 179601.70949139897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.773926108247423 > 10.389033347687384. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.705475461616352 (T) = (0 -10729.361453791156) / Math.Sqrt((15675.632838723006 / (43)) + (11912.122690404363 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.08132313801547202 = (9922.437684522789 - 10729.361453791156) / 9922.437684522789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.325578010628972 > 8.73488267449759. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.147557047109075 (T) = (0 -10210.331194965913) / Math.Sqrt((19731.906687480263 / (44)) + (24641.28230926515 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22592633772279144 = (8328.666152920756 - 10210.331194965913) / 8328.666152920756 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.365544016203714 > 41.75711515983944. IsChangePoint: Marked as a change because one of 10/24/2022 2:09:58 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.91003557970645 (T) = (0 -42435.427186521265) / Math.Sqrt((247233.8647709159 / (44)) + (833074.6227818477 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06789706309974224 = (39737.37605696343 - 42435.427186521265) / 39737.37605696343 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.38012974502112 > 9.117236929524374. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.447713385771213 (T) = (0 -9582.54083179529) / Math.Sqrt((16228.576983803301 / (44)) + (21691.583641555517 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09999683843714 = (8711.425794105035 - 9582.54083179529) / 8711.425794105035 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 184.99977656250002 > 161.42134141271112. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -58.059071857806174 (T) = (0 -184592.16956646086) / Math.Sqrt((4684168.180072664 / (44)) + (2470479.0280239466 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19586096527200555 = (154359.22312630573 - 184592.16956646086) / 154359.22312630573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.368906939421338 > 9.188267164517693. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.888427506451286 (T) = (0 -9503.642686586925) / Math.Sqrt((11108.570746577992 / (44)) + (9243.060368932338 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.087295303931036 = (8740.626996389303 - 9503.642686586925) / 8740.626996389303 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.9661069105691 > 187.2546272602983. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.85467758854346 (T) = (0 -192473.92277389133) / Math.Sqrt((3714756.136005507 / (44)) + (805300.2388315311 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07215456008515272 = (179520.68660567433 - 192473.92277389133) / 179520.68660567433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: IgnoreCase, Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 319.7862594752186 > 299.49772345279723. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.64588614895187 (T) = (0 -320244.06020716875) / Math.Sqrt((21741845.273392312 / (43)) + (11795552.620636944 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.10445452357350424 = (289956.764513044 - 320244.06020716875) / 289956.764513044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.606341909194987 > 5.5947533070289905. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.084534538329446 (T) = (0 -5716.4088990011805) / Math.Sqrt((4447.052927413562 / (44)) + (4415.30625999124 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07511458570379113 = (5317.022924825364 - 5716.4088990011805) / 5317.022924825364 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.62554162586899 > 10.391843537393914. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.36733228728902 (T) = (0 -10760.148867981963) / Math.Sqrt((23609.07576125682 / (43)) + (22480.986361031686 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.08605722172308913 = (9907.534016403297 - 10760.148867981963) / 9907.534016403297 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: None) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5629362500000001 > 1.4905887501988637. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.099114256220066 (T) = (0 -1548112.8211945556) / Math.Sqrt((160361645.48978812 / (44)) + (201077060.3466294 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08728795378598023 = (1423829.644947288 - 1548112.8211945556) / 1423829.644947288 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).MultiplyBenchmark.html>) 166.83 ns 199.43 ns 1.20 0.09 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).GreaterThanOrEqualAllBenchmark.html>) 56.96 ns 333.59 ns 5.86 0.09 False
[AbsBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).AbsBenchmark.html>) 121.19 ns 204.22 ns 1.69 0.09 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).SubtractionOperatorBenchmark.html>) 172.51 ns 183.01 ns 1.06 0.08 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).AddBenchmark.html>) 168.84 ns 201.12 ns 1.19 0.08 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanBenchmark.html>) 179.57 ns 206.84 ns 1.15 0.10 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).OnesComplementOperatorBenchmark.html>) 67.55 ns 177.02 ns 2.62 0.08 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanOrEqualBenchmark.html>) 186.64 ns 227.19 ns 1.22 0.11 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).OnesComplementBenchmark.html>) 66.75 ns 182.99 ns 2.74 0.11 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).GreaterThanOrEqualAnyBenchmark.html>) 127.47 ns 302.34 ns 2.37 0.08 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).AndNotBenchmark.html>) 168.42 ns 288.38 ns 1.71 0.14 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).NegateBenchmark.html>) 166.83 ns 201.71 ns 1.21 0.08 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).DivisionOperatorBenchmark.html>) 168.77 ns 186.30 ns 1.10 0.08 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).GreaterThanAnyBenchmark.html>) 121.78 ns 297.10 ns 2.44 0.11 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).EqualsAnyBenchmark.html>) 122.37 ns 301.32 ns 2.46 0.10 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanOrEqualAnyBenchmark.html>) 57.97 ns 270.45 ns 4.67 0.12 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).GreaterThanAllBenchmark.html>) 54.38 ns 325.37 ns 5.98 0.14 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).MultiplyOperatorBenchmark.html>) 165.91 ns 190.14 ns 1.15 0.10 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanAnyBenchmark.html>) 54.63 ns 272.19 ns 4.98 0.11 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanAllBenchmark.html>) 120.20 ns 409.27 ns 3.40 0.09 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).OneBenchmark.html>) 61.89 ns 100.58 ns 1.63 0.11 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).DivideBenchmark.html>) 168.52 ns 201.17 ns 1.19 0.12 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(Single).LessThanOrEqualAllBenchmark.html>) 127.03 ns 409.59 ns 3.22 0.12 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Single&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Single>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 199.4310164787889 > 174.271407564623. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.564316149570292 (T) = (0 -201.44629317805237) / Math.Sqrt((29.416410938671408 / (44)) + (26.187297688729128 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19951988851890695 = (167.9391022242956 - 201.44629317805237) / 167.9391022242956 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 333.58911067306497 > 59.275084601436404. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -402.54906001947916 (T) = (0 -335.4724502825259) / Math.Sqrt((0.678320724416111 / (43)) + (6.969148082694021 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -4.941085904219427 = (56.46652071539137 - 335.4724502825259) / 56.46652071539137 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.AbsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 204.22266044040472 > 126.15123518738531. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -103.46621286157664 (T) = (0 -203.06195561956076) / Math.Sqrt((11.823252261669813 / (44)) + (5.264060720195195 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6697014003306608 = (121.61573056077404 - 203.06195561956076) / 121.61573056077404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 183.00795544919978 > 173.41325217830192. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.489258370470022 (T) = (0 -183.53168264361366) / Math.Sqrt((8.306659281998535 / (44)) + (3.8254691062574833 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09974578882754027 = (166.88555165033208 - 183.53168264361366) / 166.88555165033208 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 201.11874460043754 > 175.1182987806744. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.700326367790055 (T) = (0 -201.2313380552103) / Math.Sqrt((9.680365113340892 / (44)) + (9.826211379291646 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20570308781103808 = (166.899579249272 - 201.2313380552103) / 166.899579249272 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.8444881144712 > 188.95765888528194. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.589365996355262 (T) = (0 -207.22993460159552) / Math.Sqrt((8.625795246873947 / (44)) + (28.89593498119171 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15033247920638818 = (180.14786016001477 - 207.22993460159552) / 180.14786016001477 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 177.02044382596978 > 70.54753358520205. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.44531958743733 (T) = (0 -182.6047431302439) / Math.Sqrt((1.1671253225439133 / (44)) + (71.98640626511899 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.7094068101928372 = (67.39657641786444 - 182.6047431302439) / 67.39657641786444 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 227.1917783613201 > 195.94951970749258. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.717650095600288 (T) = (0 -213.3848428676156) / Math.Sqrt((10.390023379942313 / (44)) + (17.075686355929587 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1426185462165636 = (186.75072584299903 - 213.3848428676156) / 186.75072584299903 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 182.98736577775975 > 70.84393676511652. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -156.91744781906812 (T) = (0 -179.12932899324488) / Math.Sqrt((2.0141434672062317 / (44)) + (6.887169124397505 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.6487725371100808 = (67.62729773266311 - 179.12932899324488) / 67.62729773266311 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 302.3430744506682 > 135.1347668623291. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -144.28608853887783 (T) = (0 -305.5009950455718) / Math.Sqrt((4.014995572801885 / (43)) + (21.09644563527047 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -1.3719200112789758 = (128.7990293065747 - 305.5009950455718) / 128.7990293065747 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 288.3765143806641 > 178.22712385469262. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -90.53336383392407 (T) = (0 -290.2439254948605) / Math.Sqrt((23.337738908891986 / (44)) + (17.910625085839254 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6937945970804414 = (171.35721532891174 - 290.2439254948605) / 171.35721532891174 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 201.70567924767053 > 172.82060524925572. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.99296845022934 (T) = (0 -200.47946818616563) / Math.Sqrt((12.1451102275767 / (44)) + (32.13643225603955 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21707596961552486 = (164.7222303218239 - 200.47946818616563) / 164.7222303218239 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 186.29763783934115 > 176.09835707102616. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.7385796443217 (T) = (0 -184.5043373877316) / Math.Sqrt((8.184798103754714 / (44)) + (2.756471142890703 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10062105255374014 = (167.6365693347691 - 184.5043373877316) / 167.6365693347691 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 297.0995458258374 > 127.93305610684762. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.01526569679312 (T) = (0 -303.0005707363581) / Math.Sqrt((3.848249777787215 / (44)) + (51.678254787497096 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.4727499900926329 = (122.53586976053622 - 303.0005707363581) / 122.53586976053622 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 301.3167526672282 > 127.43288475874589. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -91.02327692073656 (T) = (0 -301.4850277882831) / Math.Sqrt((59.685884803785534 / (44)) + (37.09922284249551 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.4440615723802073 = (123.35410498462801 - 301.4850277882831) / 123.35410498462801 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 270.45403138951167 > 59.27717049324619. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -107.25932060003007 (T) = (0 -273.7972598033876) / Math.Sqrt((1.617611265625806 / (44)) + (60.91769885147749 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.831589440795479 = (56.6681550985237 - 273.7972598033876) / 56.6681550985237 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 325.3658790106695 > 57.3270957070948. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -334.21173018287664 (T) = (0 -329.1292259311895) / Math.Sqrt((2.9874651716546072 / (44)) + (9.052749151721047 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.954211765734591 = (55.276708132093745 - 329.1292259311895) / 55.276708132093745 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 190.13992337098077 > 175.43935236856163. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.588600534967146 (T) = (0 -185.89771394103) / Math.Sqrt((6.70924257543575 / (44)) + (29.492853424330317 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10934585812400945 = (167.57417227427842 - 185.89771394103) / 167.57417227427842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 272.1875799467922 > 57.24907915627495. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -272.34020193094625 (T) = (0 -266.80801132768727) / Math.Sqrt((1.12799474997745 / (44)) + (8.69756033805552 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -3.860408661595671 = (54.89415189217736 - 266.80801132768727) / 54.89415189217736 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.27291051171335 > 126.65900189624166. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -134.52058434978818 (T) = (0 -408.35614409474425) / Math.Sqrt((5.865582974260899 / (44)) + (65.99798312021481 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.3486918790167715 = (121.94497399224561 - 408.35614409474425) / 121.94497399224561 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 100.57845530020548 > 64.95312354139362. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -84.05508998392992 (T) = (0 -101.46619901574874) / Math.Sqrt((1.8491902631500656 / (43)) + (2.6091285844676757 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.6282673585698488 = (62.31544130742094 - 101.46619901574874) / 62.31544130742094 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 201.17402072651117 > 175.62263728896824. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.13680496548966 (T) = (0 -201.89380013273066) / Math.Sqrt((8.633105925631243 / (44)) + (16.53280196697514 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20494916783362244 = (167.55379025300746 - 201.89380013273066) / 167.55379025300746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<Single>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 409.59080083374346 > 133.74101389675255. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -126.51119854131896 (T) = (0 -415.4299036157161) / Math.Sqrt((19.73612870799998 / (44)) + (70.18666343373391 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.221543060443863 = (128.95370194383756 - 415.4299036157161) / 128.95370194383756 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22o%22).html>) 1.43 μs 1.68 μs 1.17 0.03 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22s%22).html>) 4.84 μs 5.75 μs 1.19 0.04 False
[ParseR - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ParseR.html>) 1.27 μs 1.45 μs 1.14 0.05 False
[op_Subtraction - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.op_Subtraction.html>) 55.11 ns 61.47 ns 1.12 0.12 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22G%22).html>) 4.69 μs 5.31 μs 1.13 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22r%22).html>) 893.17 ns 1.07 μs 1.20 0.07 False
[ParseO - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ParseO.html>) 1.28 μs 1.50 μs 1.18 0.05 False
[Month - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.Month.html>) 78.52 ns 82.72 ns 1.05 0.12 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20null).html>) 4.58 μs 5.44 μs 1.19 0.03 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTime.ToString(format: "o") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.678579893722195 > 1.4976182446271442. IsChangePoint: Marked as a change because one of 10/22/2022 10:22:58 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.333462306829947 (T) = (0 -1664.5481132249079) / Math.Sqrt((3689.0361681673908 / (44)) + (347.3156848682848 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20461107676932305 = (1381.8137200673114 - 1664.5481132249079) / 1381.8137200673114 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ToString(format: "s") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.750826601369792 > 5.2077326988709265. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.422240066253018 (T) = (0 -5682.524380776451) / Math.Sqrt((10920.916254059228 / (44)) + (14021.41963686457 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1651837382228953 = (4876.9341644291 - 5682.524380776451) / 4876.9341644291 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ParseR ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4454555173692623 > 1.301683270804789. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.836009665802937 (T) = (0 -1425.4518803187061) / Math.Sqrt((658.4095584714308 / (44)) + (294.4728485511241 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15122969762859256 = (1238.199364779228 - 1425.4518803187061) / 1238.199364779228 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.op_Subtraction ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 61.47304574576756 > 58.472490128478796. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 11/15/2022 10:02:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -7.8199584078092075 (T) = (0 -61.76077818174063) / Math.Sqrt((1.5236139634380512 / (44)) + (6.88537324119285 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0976485133309384 = (56.26644361255551 - 61.76077818174063) / 56.26644361255551 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ToString(format: "G") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.308501074086605 > 4.927410850096356. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.02400190561233 (T) = (0 -5266.697552510509) / Math.Sqrt((12051.631194289603 / (44)) + (3869.4727579370388 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13421729556938133 = (4643.464328293995 - 5266.697552510509) / 4643.464328293995 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ToString(format: "r") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0682113775866047 > 926.2996832792046. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.401318661795706 (T) = (0 -1085.5723273798408) / Math.Sqrt((864.1423154345559 / (44)) + (349.4831064978809 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24313362623564197 = (873.2547366344552 - 1085.5723273798408) / 873.2547366344552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ParseO ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.50335981303522 > 1.3352944287086514. IsChangePoint: Marked as a change because one of 10/19/2022 1:55:59 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.58290905901889 (T) = (0 -1499.639454833958) / Math.Sqrt((1478.1345909430343 / (44)) + (819.8750909229417 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.16477799736052737 = (1287.4895115054126 - 1499.639454833958) / 1287.4895115054126 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.Month ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 82.71588147825771 > 82.48231477717515. IsChangePoint: Marked as a change because one of 10/19/2022 12:02:11 AM, 10/19/2022 1:55:59 PM, 12/9/2022 5:51:37 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.32641450293809 (T) = (0 -84.29946258378736) / Math.Sqrt((10.467231427087121 / (45)) + (8.465972630032484 / (14))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (14) - 2, .025) and -0.061364888280097014 = (79.42552416670911 - 84.29946258378736) / 79.42552416670911 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_DateTime.ToString(format: null) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.438167164091762 > 4.884816888450418. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.81380806102517 (T) = (0 -5433.569691450278) / Math.Sqrt((15844.463151620517 / (44)) + (56989.53026356814 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1630927001199181 = (4671.656602169424 - 5433.569691450278) / 4671.656602169424 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GreaterThanAnyBenchmark.html>) 70.90 ns 417.54 ns 5.89 0.11 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).OnesComplementOperatorBenchmark.html>) 104.03 ns 151.11 ns 1.45 0.15 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GreaterThanOrEqualAnyBenchmark.html>) 73.11 ns 412.50 ns 5.64 0.09 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).LessThanOrEqualAnyBenchmark.html>) 296.90 ns 443.37 ns 1.49 0.08 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).EqualsAnyBenchmark.html>) 266.94 ns 447.56 ns 1.68 0.15 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GreaterThanAllBenchmark.html>) 267.66 ns 488.95 ns 1.83 0.14 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).ConditionalSelectBenchmark.html>) 180.02 ns 418.56 ns 2.33 0.07 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).LessThanOrEqualAllBenchmark.html>) 73.68 ns 489.23 ns 6.64 0.04 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).AndNotBenchmark.html>) 132.56 ns 266.02 ns 2.01 0.03 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GreaterThanOrEqualAllBenchmark.html>) 281.41 ns 502.40 ns 1.79 0.05 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).LessThanAllBenchmark.html>) 71.59 ns 475.21 ns 6.64 0.06 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).SumBenchmark.html>) 136.35 ns 189.87 ns 1.39 0.07 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GetHashCodeBenchmark.html>) 1.31 μs 1.57 μs 1.20 0.02 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).OnesComplementBenchmark.html>) 103.03 ns 156.79 ns 1.52 0.13 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).LessThanAnyBenchmark.html>) 292.71 ns 428.70 ns 1.46 0.15 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).AllBitsSetBenchmark.html>) 43.78 ns 87.97 ns 2.01 0.14 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 417.54228155551044 > 74.96592814057156. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -84.46929913175069 (T) = (0 -408.8169297583714) / Math.Sqrt((5.5372822732574445 / (44)) + (236.63995663625727 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.679770557112188 = (71.97771910811649 - 408.8169297583714) / 71.97771910811649 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 151.107209823221 > 108.92193167087188. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.846931733849971 (T) = (0 -153.05047225807286) / Math.Sqrt((902.865463347637 / (44)) + (0.6666687909111989 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.254499416571371 = (122.00123032051287 - 153.05047225807286) / 122.00123032051287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 412.49753330934493 > 76.78576346228056. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -365.88028925685774 (T) = (0 -414.2297067303018) / Math.Sqrt((20.96104055970061 / (44)) + (5.757648199636679 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -4.531853412933902 = (74.88081765901472 - 414.2297067303018) / 74.88081765901472 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 443.37497170797565 > 306.195326941159. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -91.1533361242183 (T) = (0 -443.23827974975524) / Math.Sqrt((19.74024042514924 / (43)) + (35.35597740591845 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.5510619241818661 = (285.7643997569915 - 443.23827974975524) / 285.7643997569915 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 447.5618458980997 > 285.3904577065949. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.19182964710532 (T) = (0 -431.34753462889404) / Math.Sqrt((90.2258924152268 / (44)) + (27.702923917059564 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5684946418138496 = (275.00733705412733 - 431.34753462889404) / 275.00733705412733 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 488.9486617690869 > 281.6897090954013. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.734720608847354 (T) = (0 -497.292751317805) / Math.Sqrt((77.02141648724968 / (44)) + (170.85760039896326 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.818246709601993 = (273.50125188823273 - 497.292751317805) / 273.50125188823273 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 418.5625004095952 > 189.45820882664455. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.39066795056494 (T) = (0 -422.4551217476841) / Math.Sqrt((859.5646384753514 / (44)) + (84.83759691161288 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.11593866160938 = (199.65376568447655 - 422.4551217476841) / 199.65376568447655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 489.2320850500294 > 77.18349678004155. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -390.05186043498423 (T) = (0 -490.25076573482585) / Math.Sqrt((8.753249977825288 / (44)) + (14.136680522442761 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -5.666738172852123 = (73.53682610953503 - 490.25076573482585) / 73.53682610953503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.01683800783985 > 141.93565929696496. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.683238560101923 (T) = (0 -265.497697895562) / Math.Sqrt((927.3466889706323 / (44)) + (2.899121646916396 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7505900454621747 = (151.66183458187535 - 265.497697895562) / 151.66183458187535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 502.40373983670105 > 295.2874753918239. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -108.71780252807183 (T) = (0 -503.9733354379814) / Math.Sqrt((7.645693765679871 / (44)) + (59.370376155718915 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7809204370591394 = (282.9847560569298 - 503.9733354379814) / 282.9847560569298 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 475.2069716769255 > 74.74350405474941. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -196.51825001255702 (T) = (0 -478.30115245731025) / Math.Sqrt((2.2136504687299348 / (44)) + (63.49517779070692 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -5.68174379599316 = (71.58328230785098 - 478.30115245731025) / 71.58328230785098 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 189.87071392597417 > 143.3704670337081. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -111.7868949025487 (T) = (0 -188.93782270525602) / Math.Sqrt((7.220259800323417 / (44)) + (0.7654289225939728 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.37821027584870887 = (137.08925699955847 - 188.93782270525602) / 137.08925699955847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5684211260941563 > 1.381871017805964. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.780111729932585 (T) = (0 -1567.6573185922396) / Math.Sqrt((296.94973173061305 / (44)) + (341.9508485617533 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19856564865435888 = (1307.9444754254916 - 1567.6573185922396) / 1307.9444754254916 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 156.7864867801107 > 114.23848882707586. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.762202857958546 (T) = (0 -154.69879803216992) / Math.Sqrt((992.8857371213267 / (44)) + (15.189571789122466 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2694454346950218 = (121.86329069696136 - 154.69879803216992) / 121.86329069696136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 428.6968138070118 > 286.1663427769495. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -139.10696491970378 (T) = (0 -429.96072183003605) / Math.Sqrt((48.91002278972683 / (44)) + (2.2016226813255604 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5696877802225211 = (273.91480474485456 - 429.96072183003605) / 273.91480474485456 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int16>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 87.97013529170391 > 46.433928421125714. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.604727911121131 (T) = (0 -88.04561552240499) / Math.Sqrt((886.4859274449992 / (44)) + (5.990671362869441 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.40557708866896813 = (62.64018973571992 - 88.04561552240499) / 62.64018973571992 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Equals_Valid - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid.html>) 413.10 ns 546.67 ns 1.32 0.10 False
[LastIndexOf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf.html>) 157.39 ns 206.04 ns 1.31 0.12 False
[Equals_Object_Valid - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid.html>) 453.92 ns 631.96 ns 1.39 0.06 False
[TrimEnd - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd.html>) 592.38 ns 752.84 ns 1.27 0.05 False
[IndexOfAny - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny.html>) 195.66 ns 290.84 ns 1.49 0.11 False
[TrimStart - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart.html>) 549.67 ns 672.17 ns 1.22 0.04 False
[Indexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer.html>) 568.18 ns 608.90 ns 1.07 0.04 False
[IndexOf - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf.html>) 221.79 ns 252.67 ns 1.14 0.14 False
[StartsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith.html>) 237.94 ns 317.36 ns 1.33 0.19 False
[EndsWith - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith.html>) 275.23 ns 309.57 ns 1.12 0.21 False
[GetSegmentHashCode - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode.html>) 255.23 ns 292.27 ns 1.15 0.18 False
[Equals_String - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String.html>) 300.81 ns 467.35 ns 1.55 0.14 False

graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 546.6707650374317 > 431.9979369751534. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.49779082786493 (T) = (0 -555.1798248273274) / Math.Sqrt((60.42030571717567 / (44)) + (248.64397808430968 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35732910504734167 = (409.02373843074963 - 555.1798248273274) / 409.02373843074963 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 206.03692195657106 > 165.28806835884114. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.94352524323863 (T) = (0 -205.67339400407954) / Math.Sqrt((28.412727824431887 / (44)) + (2.2839873009980822 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2913381468888231 = (159.27152349646096 - 205.67339400407954) / 159.27152349646096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 631.9587684865578 > 480.1110046308706. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.15230102936661 (T) = (0 -624.4198096160759) / Math.Sqrt((138.063844051346 / (44)) + (126.53609205793171 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3556483474096882 = (460.60603460269704 - 624.4198096160759) / 460.60603460269704 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 752.8428093660362 > 621.7656658429055. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.72028728444984 (T) = (0 -760.2564291593288) / Math.Sqrt((45.328720012320396 / (44)) + (170.81361104959768 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2764130020808852 = (595.6194647969843 - 760.2564291593288) / 595.6194647969843 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 290.83976393834445 > 204.62753380675522. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.853963552174385 (T) = (0 -292.0916753350117) / Math.Sqrt((21.754119440919684 / (44)) + (56.250641631472384 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.49359534010917083 = (195.56279233815897 - 292.0916753350117) / 195.56279233815897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 672.1717910825512 > 572.5858979130261. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.340282318653008 (T) = (0 -695.5832868938261) / Math.Sqrt((73.2393912762295 / (44)) + (2407.525453333505 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2918574289165803 = (538.43657304907 - 695.5832868938261) / 538.43657304907 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 608.9009597610064 > 590.0765421992753. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.63160074804484 (T) = (0 -607.1264135540722) / Math.Sqrt((82.95573497160134 / (43)) + (31.58118110217684 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.09050873193119582 = (556.7368658102391 - 607.1264135540722) / 556.7368658102391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 252.66614933782373 > 215.8601414586776. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.294480191819158 (T) = (0 -257.63943379024516) / Math.Sqrt((35.216395876078074 / (44)) + (48.852953441532165 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25876989313295484 = (204.67556079610856 - 257.63943379024516) / 204.67556079610856 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 317.3632697594561 > 254.534430543139. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.503607960282714 (T) = (0 -327.9625217138593) / Math.Sqrt((42.09950781176897 / (44)) + (120.36478351232462 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35219187562436116 = (242.54140823204241 - 327.9625217138593) / 242.54140823204241 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 309.5662366654731 > 261.3759889212529. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.0417045852242 (T) = (0 -316.3642932277477) / Math.Sqrt((55.89490304530315 / (44)) + (125.24352347949164 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2599065523970758 = (251.1013952787519 - 316.3642932277477) / 251.1013952787519 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 292.26638840841986 > 273.2027173166892. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.72965198638073 (T) = (0 -294.44453526060454) / Math.Sqrt((60.97539395866547 / (44)) + (105.7172071684467 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1308227023268854 = (260.3808135923768 - 294.44453526060454) / 260.3808135923768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 467.34734674669295 > 325.9065857844057. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.49115150752899 (T) = (0 -468.79742492953756) / Math.Sqrt((66.02466446870382 / (44)) + (118.58073061512316 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.538327746870269 = (304.7448281962715 - 468.79742492953756) / 304.7448281962715 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonReflection.html>) 17.91 ms 20.63 ms 1.15 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToString(MyEventsListerViewModel).JsonNet.html>) 19.93 ms 22.07 ms 1.11 0.00 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonSourceGen.html>) 16.57 ms 18.95 ms 1.14 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.62651112777778 > 18.9065596125. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.11954640823168 (T) = (0 -20505091.74859788) / Math.Sqrt((36060035919.68814 / (44)) + (26301980914.345436 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14153190298591958 = (17962784.653641697 - 20505091.74859788) / 17962784.653641697 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 22.066307142857138 > 21.14097362931818. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.028085579637704 (T) = (0 -22135962.572727717) / Math.Sqrt((110245268876.32031 / (44)) + (13599824025.521143 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09865510968130656 = (20148236.12766779 - 22135962.572727717) / 20148236.12766779 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.950825651282052 > 17.40253783566667. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.32667313168289 (T) = (0 -18872575.652883444) / Math.Sqrt((49555944525.400116 / (44)) + (5138963360.033557 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14491278845461114 = (16483854.353969973 - 18872575.652883444) / 16483854.353969973 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Booleans

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBooleans - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 46.95 ms 50.76 ms 1.08 0.09 False
[WriteBooleans - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 32.84 ms 34.89 ms 1.06 0.10 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 50.757750066666674 > 49.854131305625. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.379507188880709 (T) = (0 -51951885.55902931) / Math.Sqrt((1737931006655.8757 / (44)) + (950450854417.5518 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09003386640955557 = (47660799.50355374 - 51951885.55902931) / 47660799.50355374 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.891428612244894 > 34.84010115538462. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.447331388589646 (T) = (0 -35404688.43348683) / Math.Sqrt((323006649316.95013 / (44)) + (552119609773.6484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05939746434601343 = (33419646.19043413 - 35404688.43348683) / 33419646.19043413 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Cache

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%2040000%2c%20unique%3a%201600%2c%20cacheSize%3a%2015).html>) 3.35 secs 3.54 secs 1.06 0.00 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%2040000%2c%20unique%3a%201600%2c%20cacheSize%3a%203200).html>) 278.37 ms 298.95 ms 1.07 0.03 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%20400000%2c%20unique%3a%201%2c%20cacheSize%3a%2015).html>) 1.21 secs 1.32 secs 1.09 0.07 False
[IsMatch - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total%3a%2040000%2c%20unique%3a%201600%2c%20cacheSize%3a%20800).html>) 2.22 secs 2.34 secs 1.05 0.00 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Cache*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 15) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5415275831666664 > 3.522697722008654. IsChangePoint: Marked as a change because one of 10/19/2022 4:51:05 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.404231320604683 (T) = (0 -3553591029.083406) / Math.Sqrt((1331531011002755.5 / (44)) + (847084770992267 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06237889742860845 = (3344937514.93422 - 3553591029.083406) / 3344937514.93422 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 3200) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 298.95180006666664 > 292.757027355. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.946529501788635 (T) = (0 -302151695.7371306) / Math.Sqrt((19783452802532.566 / (44)) + (28569736852149.94 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08811378843365376 = (277683914.0804196 - 302151695.7371306) / 277683914.0804196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 400000, unique: 1, cacheSize: 15) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.3175930665333333 > 1.2830307703850001. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.815206068779595 (T) = (0 -1343470982.2088888) / Math.Sqrt((561942445602575.25 / (43)) + (235199543303688.8 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1050282855872433 = (1215779722.3217053 - 1343470982.2088888) / 1215779722.3217053 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.RegularExpressions.Tests.Perf_Regex_Cache.IsMatch(total: 40000, unique: 1600, cacheSize: 800) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.3380126664666667 > 2.3337593301050004. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.666410992732587 (T) = (0 -2349449392.095238) / Math.Sqrt((316669714742478.3 / (44)) + (263133786053210.03 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.057458741445849074 = (2221788236.2794294 - 2349449392.095238) / 2221788236.2794294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateObject

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase%3a%20ObjectProperties).html>) 21.53 ms 29.09 ms 1.35 0.03 False
[PropertyIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase%3a%20StringProperties).html>) 135.21 μs 214.55 μs 1.59 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase%3a%20NumericProperties).html>) 571.33 μs 718.07 μs 1.26 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase%3a%20StringProperties).html>) 452.32 μs 599.34 μs 1.33 0.04 False
[EnumerateProperties - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase%3a%20ObjectProperties).html>) 98.40 μs 120.62 μs 1.23 0.09 False
[PropertyIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase%3a%20NumericProperties).html>) 135.57 μs 214.46 μs 1.58 0.05 False
[EnumerateProperties - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase%3a%20StringProperties).html>) 95.15 μs 116.76 μs 1.23 0.06 False
[PropertyIndexer - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase%3a%20ObjectProperties).html>) 141.67 μs 237.66 μs 1.68 0.08 False
[EnumerateProperties - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase%3a%20NumericProperties).html>) 95.16 μs 117.24 μs 1.23 0.10 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateObject*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: ObjectProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.087103166666665 > 22.649114087727273. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.07745785550355 (T) = (0 -29007190.37761023) / Math.Sqrt((196457307680.5946 / (44)) + (69141267052.18654 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35078518612898035 = (21474317.80825028 - 29007190.37761023) / 21474317.80825028 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: StringProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.54866866438357 > 143.50750572156832. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -119.43760079189364 (T) = (0 -216600.1035473261) / Math.Sqrt((8587040.051172059 / (44)) + (3760997.9287952036 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5828092022186094 = (136845.36534392124 - 216600.1035473261) / 136845.36534392124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: NumericProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 718.0744721996754 > 591.1823783680557. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.70867425316838 (T) = (0 -733296.8527283596) / Math.Sqrt((629409699.6323406 / (44)) + (170616351.79013655 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29204642492175864 = (567546.8300396135 - 733296.8527283596) / 567546.8300396135 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.Parse(TestCase: StringProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 599.3373392628207 > 483.2683990909091. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.08738296198569 (T) = (0 -596278.0410393794) / Math.Sqrt((142654126.27158716 / (44)) + (110661311.4200244 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2805486911848254 = (465642.6148760296 - 596278.0410393794) / 465642.6148760296 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: ObjectProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 120.61850903159338 > 103.05572026069542. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.84599889261933 (T) = (0 -122517.84151332392) / Math.Sqrt((4132008.1021941747 / (44)) + (4400134.914529761 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23797349834807865 = (98966.44934387425 - 122517.84151332392) / 98966.44934387425 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: NumericProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.4559077177104 > 142.4619598154634. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -86.86353766787884 (T) = (0 -214447.74544028146) / Math.Sqrt((10182623.117181575 / (44)) + (8438069.438447041 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5647589380359112 = (137048.42338811417 - 214447.74544028146) / 137048.42338811417 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: StringProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 116.76435844216418 > 100.20375252996811. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.054742317625646 (T) = (0 -117937.02578184784) / Math.Sqrt((3071553.1288353675 / (44)) + (4583335.115847723 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23046672749215408 = (95847.39119457405 - 117937.02578184784) / 95847.39119457405 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.PropertyIndexer(TestCase: ObjectProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 237.6560545410156 > 146.34197562235798. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -65.68142702806516 (T) = (0 -221805.74859048653) / Math.Sqrt((5025026.363303204 / (44)) + (21127310.990312595 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5758111325299773 = (140756.5564246114 - 221805.74859048653) / 140756.5564246114 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Document.Tests.Perf_EnumerateObject.EnumerateProperties(TestCase: NumericProperties) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 117.23927242137525 > 100.03593441030891. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.138399362928688 (T) = (0 -117743.55315460295) / Math.Sqrt((1022195.7318276567 / (44)) + (8024989.2707292875 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23646835443819184 = (95225.69076027953 - 117743.55315460295) / 95225.69076027953 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).Span(Size%3a%20512).html>) 7.55 ms 12.09 ms 1.60 0.06 False
[Queue - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).Queue(Size%3a%20512).html>) 16.58 ms 20.95 ms 1.26 0.08 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).Array(Size%3a%20512).html>) 16.25 ms 21.05 ms 1.30 0.05 False
[Stack - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).Stack(Size%3a%20512).html>) 16.31 ms 21.53 ms 1.32 0.04 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).ImmutableArray(Size%3a%20512).html>) 16.16 ms 21.65 ms 1.34 0.08 False
[LinkedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).LinkedList(Size%3a%20512).html>) 16.64 ms 21.54 ms 1.29 0.08 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).List(Size%3a%20512).html>) 16.26 ms 21.53 ms 1.32 0.08 False
[SortedSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).SortedSet(Size%3a%20512).html>) 2.75 ms 3.23 ms 1.17 0.08 False
[ImmutableList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).ImmutableList(Size%3a%20512).html>) 34.01 ms 39.64 ms 1.17 0.06 False
[ICollection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsFalse(String).ICollection(Size%3a%20512).html>) 16.29 ms 21.02 ms 1.29 0.04 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsFalse<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.08814511830357 > 7.902124196093749. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.44404624695208 (T) = (0 -11087569.448121186) / Math.Sqrt((13453436163.563612 / (44)) + (125870867918.23563 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4778825412357033 = (7502334.684088308 - 11087569.448121186) / 7502334.684088308 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Queue(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.946733333333327 > 17.189842838. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.03264765926006 (T) = (0 -21129115.710625485) / Math.Sqrt((47519545103.354706 / (44)) + (19427352988.02553 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29218621104440706 = (16351448.057589097 - 21129115.710625485) / 16351448.057589097 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.05450595238095 > 17.042253856923075. IsChangePoint: Marked as a change because one of 10/10/2022 3:36:43 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.63716108074637 (T) = (0 -21108458.16084175) / Math.Sqrt((47592058964.5363 / (44)) + (62354256123.72841 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2961593942144577 = (16285387.626754511 - 21108458.16084175) / 16285387.626754511 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.Stack(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.529259733766235 > 17.15890800716667. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.09603727296745 (T) = (0 -21086562.275828615) / Math.Sqrt((54871541879.901276 / (44)) + (69093548876.85503 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2848032256542669 = (16412289.333326196 - 21086562.275828615) / 16412289.333326196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.65468484848485 > 17.16154882316667. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.36179055279029 (T) = (0 -21155035.581796356) / Math.Sqrt((269714947871.38712 / (44)) + (48449997522.969315 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28397527537778733 = (16476201.674188668 - 21155035.581796356) / 16476201.674188668 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.LinkedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.54253895454545 > 17.422199676. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -79.90947351789053 (T) = (0 -21630834.58429793) / Math.Sqrt((92385296790.59448 / (44)) + (24598481289.584766 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29183949959082006 = (16744212.103089683 - 21630834.58429793) / 16744212.103089683 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.5290484969697 > 17.024364340833333. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.38548030509095 (T) = (0 -21191103.352548007) / Math.Sqrt((110212693530.37613 / (44)) + (92545425566.91089 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29910579899784595 = (16312068.939185102 - 21191103.352548007) / 16312068.939185102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.SortedSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.2306691650000006 > 2.8620409368229165. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -48.24127658903573 (T) = (0 -3165439.7560592187) / Math.Sqrt((1091222697.2759454 / (44)) + (832722889.0553286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15818361737347822 = (2733107.0035663117 - 3165439.7560592187) / 2733107.0035663117 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.ImmutableList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 39.644642845238096 > 35.924669281071424. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.0097576705923 (T) = (0 -39559732.675604396) / Math.Sqrt((336239357779.26324 / (44)) + (71334906180.80258 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1600143777204466 = (34102795.13374958 - 39559732.675604396) / 34102795.13374958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsFalse<String>.ICollection(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.021791666666665 > 17.04855999730769. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -84.30046570270743 (T) = (0 -21148870.23825064) / Math.Sqrt((40998636318.04153 / (44)) + (36153682268.84616 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.29943512987538307 = (16275433.64960345 - 21148870.23825064) / 16275433.64960345 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EmptyStaticFunction5Arg - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg.html>) 30.12 ms 34.12 ms 1.13 0.07 False
[StructWithInterfaceInterfaceMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod.html>) 3.17 ms 3.37 ms 1.06 0.09 False
[GenericClassGenericStaticMethod - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod.html>) 3.08 ms 3.27 ms 1.06 0.09 False
[ForeachOverList100Elements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.LowLevelPerf.ForeachOverList100Elements.html>) 516.92 ms 574.69 ms 1.11 0.09 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.LowLevelPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.LowLevelPerf.EmptyStaticFunction5Arg ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.12014286904762 > 31.607353732500002. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 11/11/2022 3:15:52 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.051696752622384 (T) = (0 -34098728.497385316) / Math.Sqrt((553782272315.6227 / (44)) + (112895204657.47264 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12151624801272289 = (30404132.403615866 - 34098728.497385316) / 30404132.403615866 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.StructWithInterfaceInterfaceMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3734158325 > 3.3439847566947116. IsChangePoint: Marked as a change because one of 11/17/2022 7:01:39 PM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.64832160731333 (T) = (0 -3392785.0393099813) / Math.Sqrt((3369343402.37963 / (43)) + (4518475341.097805 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.058655196085649126 = (3204806.486431766 - 3392785.0393099813) / 3204806.486431766 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.271392858928572 > 3.234083639166667. IsChangePoint: Marked as a change because one of 11/16/2022 11:33:39 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.710978100215213 (T) = (0 -3316647.7063547) / Math.Sqrt((2291695514.1363316 / (44)) + (8364973885.131793 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0779348419851712 = (3076853.6067046677 - 3316647.7063547) / 3076853.6067046677 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.LowLevelPerf.ForeachOverList100Elements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 574.6925332666666 > 531.7693043007692. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.02217489086755 (T) = (0 -557627097.4104762) / Math.Sqrt((160478192395050.06 / (44)) + (71946656395776.9 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09109760642199122 = (511069856.7464451 - 557627097.4104762) / 511069856.7464451 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.CtorFromCollectionNonGeneric<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CtorFromCollectionNonGeneric(String).SortedList(Size%3a%20512).html>) 3.28 ms 3.78 ms 1.15 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CtorFromCollectionNonGeneric<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.7779391666666666 > 3.46454166028125. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.375513328767276 (T) = (0 -3771131.037700397) / Math.Sqrt((3418224365.8405004 / (44)) + (1119872954.5974097 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13937924674323532 = (3309811.942318307 - 3771131.037700397) / 3309811.942318307 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).Array_ComparerClass(Size%3a%20512).html>) 308.40 μs 342.23 μs 1.11 0.04 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).List(Size%3a%20512).html>) 174.90 μs 315.95 μs 1.81 0.04 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).Array_Comparison(Size%3a%20512).html>) 308.67 μs 340.96 μs 1.10 0.04 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).Array(Size%3a%20512).html>) 174.11 μs 316.39 μs 1.82 0.06 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(Int32).Array_ComparerStruct(Size%3a%20512).html>) 348.88 μs 379.86 μs 1.09 0.03 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<Int32>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 342.22558466153845 > 323.39120253749996. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.25732869934548 (T) = (0 -344867.245017436) / Math.Sqrt((11324566.31129964 / (44)) + (31163157.062200475 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12042288511473882 = (307800.96479564434 - 344867.245017436) / 307800.96479564434 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<Int32>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 315.9510429142857 > 185.30824414684614. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -100.81110992924349 (T) = (0 -319034.2495701733) / Math.Sqrt((7249003.61345435 / (44)) + (27649057.1232544 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8108402672098558 = (176180.22712833836 - 319034.2495701733) / 176180.22712833836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<Int32>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 340.96264 > 321.72630221199995. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.224740146147337 (T) = (0 -342879.5846400781) / Math.Sqrt((16464182.00739465 / (44)) + (18454773.00050949 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11182215640054707 = (308394.27211104403 - 342879.5846400781) / 308394.27211104403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<Int32>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 316.3948571 > 183.09238499900002. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -129.93358542877976 (T) = (0 -317266.7076808279) / Math.Sqrt((9426618.85103051 / (44)) + (14659678.064126654 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8084735035954685 = (175433.4288282701 - 317266.7076808279) / 175433.4288282701 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<Int32>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 379.8568769384614 > 351.6587728534615. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.411771380734855 (T) = (0 -371138.1785028913) / Math.Sqrt((20760700.238847606 / (44)) + (13668680.15277041 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10277722840752176 = (336548.64186744014 - 371138.1785028913) / 336548.64186744014 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse3 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryParse3.html>) 1.42 μs 1.54 μs 1.08 0.08 False
[Parse4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.Parse4.html>) 1.96 μs 2.17 μs 1.11 0.08 False
[ToStringL - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.ToStringL.html>) 2.40 μs 3.01 μs 1.25 0.07 False
[Parse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.Parse2.html>) 1.06 μs 1.18 μs 1.12 0.06 False
[ToString2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.ToString2.html>) 1.04 μs 1.11 μs 1.07 0.07 False
[TryParse2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryParse2.html>) 1.01 μs 1.12 μs 1.10 0.07 False
[TryFormatL - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryFormatL.html>) 2.03 μs 2.47 μs 1.22 0.06 False
[TryParse4 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Version.TryParse4.html>) 1.89 μs 2.02 μs 1.07 0.06 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Version.TryParse3 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5408787599066427 > 1.5026261252956525. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/1/2022 5:03:18 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.741044065392476 (T) = (0 -1571.7964484595252) / Math.Sqrt((907.2641919472441 / (44)) + (492.7515319615149 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0844605789064779 = (1449.380898699384 - 1571.7964484595252) / 1449.380898699384 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.Parse4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1749707862220315 > 2.087405700065444. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/1/2022 7:29:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.285064779767287 (T) = (0 -2177.4860865223213) / Math.Sqrt((2149.855762560285 / (44)) + (12182.757752950769 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0925341675659293 = (1993.0599437210922 - 2177.4860865223213) / 1993.0599437210922 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToStringL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0141981948373404 > 2.5210678423540815. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.530273982600434 (T) = (0 -3000.3673317739167) / Math.Sqrt((3319.042804784498 / (44)) + (603.2330290991484 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24292892086229312 = (2413.949246343375 - 3000.3673317739167) / 2413.949246343375 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.Parse2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.179229136747416 > 1.1189777987506724. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.42116216920777 (T) = (0 -1156.2304305622986) / Math.Sqrt((520.56623213681 / (44)) + (321.7787149621499 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08335975129889404 = (1067.263602119274 - 1156.2304305622986) / 1067.263602119274 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.ToString2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.11080542161637 > 1.1005218784295792. IsChangePoint: Marked as a change because one of 10/3/2022 1:11:02 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.045605969051513 (T) = (0 -1124.7483374842511) / Math.Sqrt((1073.6301618687291 / (44)) + (622.2183121554914 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0781604894770351 = (1043.210494598827 - 1124.7483374842511) / 1043.210494598827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryParse2 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1184307083833975 > 1.0606109839178355. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.42557878277962 (T) = (0 -1107.064550083906) / Math.Sqrt((370.14266713559675 / (44)) + (163.70483766178603 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09299243985750164 = (1012.8748468088571 - 1107.064550083906) / 1012.8748468088571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryFormatL ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.4701805738159353 > 2.0572331412582745. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.09963603525205 (T) = (0 -2468.743303435054) / Math.Sqrt((738.473620546196 / (43)) + (614.3662396319633 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.25650313523797075 = (1964.7728956661101 - 2468.743303435054) / 1964.7728956661101 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Version.TryParse4 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.024198709681522 > 1.9932602953035983. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.04496388228115 (T) = (0 -2032.1434548759146) / Math.Sqrt((1279.9260017685979 / (44)) + (190.00648755039148 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0753841052015092 = (1889.6908044731836 - 2032.1434548759146) / 1889.6908044731836 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SinPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.SinPi.html>) 605.05 μs 696.44 μs 1.15 0.06 False
[Hypot - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Hypot.html>) 395.30 μs 560.32 μs 1.42 0.10 False
[SinCosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.SinCosPi.html>) 1.31 ms 1.50 ms 1.14 0.08 False
[CosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.CosPi.html>) 613.82 μs 697.28 μs 1.14 0.13 False
[ILogB - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.ILogB.html>) 248.37 μs 270.97 μs 1.09 0.03 False
[TanPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.TanPi.html>) 587.97 μs 679.98 μs 1.16 0.06 False
[CopySign - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.CopySign.html>) 220.47 μs 290.45 μs 1.32 0.08 False
[Round - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Round.html>) 316.01 μs 507.30 μs 1.61 0.10 False
[Abs - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.MathBenchmarks.Single.Abs.html>) 64.22 μs 112.63 μs 1.75 0.07 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.SinPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 696.4398288043478 > 634.3312363119452. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.476212287576395 (T) = (0 -704130.729042612) / Math.Sqrt((108208511.30599554 / (44)) + (58492377.65949578 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15981370521056007 = (607106.7498851288 - 704130.729042612) / 607106.7498851288 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.Hypot ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 560.315643121302 > 416.644484375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.40128975864758 (T) = (0 -568506.8825720187) / Math.Sqrt((201933628.24390495 / (44)) + (536193361.2869852 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4183191924278501 = (400831.4105930274 - 568506.8825720187) / 400831.4105930274 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.SinCosPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.5044291678030304 > 1.3708384371354168. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.90655067293028 (T) = (0 -1497369.8318702132) / Math.Sqrt((475978196.1989343 / (44)) + (249670953.4790695 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13916640198598676 = (1314443.464325972 - 1497369.8318702132) / 1314443.464325972 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.CosPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 697.2788469899665 > 632.6853063040866. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.45541575994384 (T) = (0 -696122.2318840668) / Math.Sqrt((181070385.7665322 / (44)) + (20081299.84325983 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.143715749108643 = (608649.6862761495 - 696122.2318840668) / 608649.6862761495 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.ILogB ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 270.968976846247 > 262.0934503673835. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.595242130892107 (T) = (0 -272086.593098085) / Math.Sqrt((5756614.571964284 / (44)) + (36253964.75394663 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08666744313535615 = (250386.25645490485 - 272086.593098085) / 250386.25645490485 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.TanPi ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 679.9829192546584 > 615.3112845543982. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.23570312128166 (T) = (0 -702828.0236718747) / Math.Sqrt((105237152.98158918 / (44)) + (1223407687.4209542 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1897863923067907 = (590717.8197837784 - 702828.0236718747) / 590717.8197837784 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.CopySign ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 290.4456017691799 > 233.12606526408453. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -62.13117538612544 (T) = (0 -283111.7509514131) / Math.Sqrt((18071912.839217864 / (44)) + (8064345.124300288 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2718039615688419 = (222606.43896892632 - 283111.7509514131) / 222606.43896892632 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.Round ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 507.2980993663594 > 333.4567283149039. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.94798862023464 (T) = (0 -511353.8957838328) / Math.Sqrt((12956823.884701155 / (43)) + (52231070.37891158 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.6035396874889797 = (318890.7014734221 - 511353.8957838328) / 318890.7014734221 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.MathBenchmarks.Single.Abs ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 112.63008435943554 > 67.59489688390315. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -213.63244866089786 (T) = (0 -112845.62793319178) / Math.Sqrt((860929.6882733614 / (44)) + (462639.000253487 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7392736631570069 = (64880.89271032964 - 112845.62793319178) / 64880.89271032964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt16>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanOrEqualAllBenchmark.html>) 206.54 ns 521.49 ns 2.52 0.09 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).OnesComplementBenchmark.html>) 67.16 ns 228.15 ns 3.40 0.09 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).DivideBenchmark.html>) 264.35 ns 347.98 ns 1.32 0.08 False
[DotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).DotBenchmark.html>) 218.67 ns 304.71 ns 1.39 0.11 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).AddBenchmark.html>) 247.37 ns 332.39 ns 1.34 0.07 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanAnyBenchmark.html>) 51.47 ns 368.15 ns 7.15 0.07 False
[MultiplyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).MultiplyBenchmark.html>) 244.51 ns 331.47 ns 1.36 0.10 False
[GreaterThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).GreaterThanOrEqualBenchmark.html>) 279.14 ns 326.34 ns 1.17 0.11 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).EqualsAnyBenchmark.html>) 194.00 ns 426.67 ns 2.20 0.10 False
[UnaryNegateOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).UnaryNegateOperatorBenchmark.html>) 246.48 ns 331.00 ns 1.34 0.07 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).GreaterThanOrEqualAllBenchmark.html>) 53.20 ns 472.05 ns 8.87 0.08 False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).SquareRootBenchmark.html>) 229.37 ns 278.48 ns 1.21 0.13 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).GreaterThanAnyBenchmark.html>) 193.35 ns 426.74 ns 2.21 0.05 False
[DivisionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).DivisionOperatorBenchmark.html>) 273.22 ns 333.88 ns 1.22 0.06 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).AndNotBenchmark.html>) 170.06 ns 344.23 ns 2.02 0.07 False
[LessThanOrEqualBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanOrEqualBenchmark.html>) 271.89 ns 324.46 ns 1.19 0.08 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).OnesComplementOperatorBenchmark.html>) 66.81 ns 230.52 ns 3.45 0.09 False
[EqualityOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).EqualityOperatorBenchmark.html>) 51.01 ns 54.61 ns 1.07 0.13 False
[OneBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).OneBenchmark.html>) 89.49 ns 164.25 ns 1.84 0.08 False
[LessThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanBenchmark.html>) 262.90 ns 310.77 ns 1.18 0.10 False
[MaxBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).MaxBenchmark.html>) 293.97 ns 348.03 ns 1.18 0.07 False
[EqualsBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).EqualsBenchmark.html>) 83.22 ns 93.53 ns 1.12 0.12 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).SubtractBenchmark.html>) 243.98 ns 332.29 ns 1.36 0.13 False
[MinBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).MinBenchmark.html>) 302.93 ns 350.32 ns 1.16 0.09 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanAllBenchmark.html>) 193.13 ns 502.73 ns 2.60 0.08 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).LessThanOrEqualAnyBenchmark.html>) 53.44 ns 380.20 ns 7.11 0.11 False
[EqualsAllBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).EqualsAllBenchmark.html>) 51.39 ns 55.61 ns 1.08 0.11 False
[EqualsStaticBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).EqualsStaticBenchmark.html>) 264.04 ns 317.31 ns 1.20 0.09 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).GreaterThanOrEqualAnyBenchmark.html>) 206.07 ns 448.25 ns 2.18 0.09 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).AddOperatorBenchmark.html>) 246.58 ns 315.25 ns 1.28 0.09 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).NegateBenchmark.html>) 250.57 ns 345.11 ns 1.38 0.08 False
[GreaterThanBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).GreaterThanBenchmark.html>) 266.68 ns 316.53 ns 1.19 0.09 False
[MultiplyOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).MultiplyOperatorBenchmark.html>) 245.67 ns 318.72 ns 1.30 0.09 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).ConditionalSelectBenchmark.html>) 347.13 ns 505.42 ns 1.46 0.09 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.blob.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_VectorOf(UInt16).SubtractionOperatorBenchmark.html>) 246.37 ns 314.39 ns 1.28 0.07 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt16&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 521.4900812775468 > 216.47494948716317. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -204.77679382332647 (T) = (0 -522.2539554147245) / Math.Sqrt((29.25691296993352 / (44)) + (25.366932727573243 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5116493440895327 = (207.93267047555972 - 522.2539554147245) / 207.93267047555972 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 228.15416531792528 > 70.50011893585011. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -101.12355003143686 (T) = (0 -232.5001557317037) / Math.Sqrt((1.8114118150493224 / (43)) + (36.66431985978332 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -2.4423325434750476 = (67.54145707752974 - 232.5001557317037) / 67.54145707752974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 347.98241769024 > 278.03113194773465. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.25374084779907 (T) = (0 -348.4817537351153) / Math.Sqrt((49.40872322122366 / (44)) + (4.945543849747302 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.30630165265391135 = (266.7697411444991 - 348.4817537351153) / 266.7697411444991 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.DotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 304.71443915663764 > 229.10118380238038. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.726017359224215 (T) = (0 -298.2638998601285) / Math.Sqrt((9.065004613573382 / (44)) + (28.874169464601056 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36582872493369956 = (218.37577026695416 - 298.2638998601285) / 218.37577026695416 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 332.38878620022217 > 259.52362174409484. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -116.7482327949585 (T) = (0 -331.6083652763962) / Math.Sqrt((17.273657113843306 / (44)) + (1.9189416708095663 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3404896447221811 = (247.37853558363196 - 331.6083652763962) / 247.37853558363196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 368.15481693617977 > 54.0521182828861. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -880.8102451182865 (T) = (0 -366.8383925969043) / Math.Sqrt((1.3335294883391406 / (44)) + (1.4654284893060594 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -6.094588426142222 = (51.70678981816253 - 366.8383925969043) / 51.70678981816253 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.MultiplyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 331.4677269768955 > 257.7901539388036. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.510418963966067 (T) = (0 -332.9152620609641) / Math.Sqrt((334.0530762697665 / (44)) + (35.04848003728162 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3349640807409333 = (249.381437945648 - 332.9152620609641) / 249.381437945648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 326.33878231196655 > 293.47309633426323. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.503415291879232 (T) = (0 -331.71462000633585) / Math.Sqrt((43.5278447316417 / (44)) + (53.62134380129181 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17837012348706718 = (281.5029110078897 - 331.71462000633585) / 281.5029110078897 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 426.6734794367522 > 203.43147952437405. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -131.3708625534234 (T) = (0 -429.3287603976551) / Math.Sqrt((10.34946736938524 / (44)) + (44.30044270481144 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.204566593783255 = (194.74519917354112 - 429.3287603976551) / 194.74519917354112 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.UnaryNegateOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 330.99622009159594 > 255.67864327637997. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -158.9919397383374 (T) = (0 -329.3916892743928) / Math.Sqrt((7.5344905808282885 / (43)) + (1.532330726740575 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.34086460869780033 = (245.65618865448778 - 329.3916892743928) / 245.65618865448778 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 472.05314363133823 > 55.78523738903119. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -456.7427890029853 (T) = (0 -474.5832464109066) / Math.Sqrt((11.670366020884801 / (44)) + (8.733860828430627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -7.770929919626341 = (54.10865789144565 - 474.5832464109066) / 54.10865789144565 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.SquareRootBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 278.4806123026108 > 240.5768133413709. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.915961963050478 (T) = (0 -282.2409582311889) / Math.Sqrt((36.445989791814036 / (43)) + (46.18050952229686 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.22241038368721378 = (230.88887496182116 - 282.2409582311889) / 230.88887496182116 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 426.74198177346494 > 203.5285176078577. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -106.05558257320189 (T) = (0 -431.76771225109286) / Math.Sqrt((7.370022541744529 / (44)) + (72.06329322031654 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.2108847460091343 = (195.29182289148102 - 431.76771225109286) / 195.29182289148102 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.DivisionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 333.8827555345155 > 279.5054504714517. IsChangePoint: Marked as a change because one of 9/28/2022 3:03:11 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -154.69597143370478 (T) = (0 -331.008196308395) / Math.Sqrt((4.298734526809279 / (44)) + (1.1803926653762322 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2442164017995147 = (266.03748015992767 - 331.008196308395) / 266.03748015992767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 344.2304895551523 > 178.5112107251091. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.349698300457 (T) = (0 -344.5239505308612) / Math.Sqrt((20.673601857543094 / (44)) + (39.44468903047101 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.0103467213924737 = (171.37538856592136 - 344.5239505308612) / 171.37538856592136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 324.4644859269108 > 287.1590219465213. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.082947328697749 (T) = (0 -327.7717069914083) / Math.Sqrt((30.98323220696611 / (44)) + (234.31107598046495 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19228188828297454 = (274.9112522907128 - 327.7717069914083) / 274.9112522907128 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 230.51834644136213 > 70.01029602872006. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -135.36154729731734 (T) = (0 -230.83551917496894) / Math.Sqrt((7.347385807243965 / (43)) + (18.1789683887441 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -2.458018641380783 = (66.75369427239309 - 230.83551917496894) / 66.75369427239309 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualityOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 54.610582677011415 > 53.57857218409743. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 12/9/2022 9:29:01 AM, 12/9/2022 7:57:12 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -4.512004160631313 (T) = (0 -55.84588711706426) / Math.Sqrt((12.05202219627578 / (46)) + (4.03715727684486 / (13))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (13) - 2, .025) and -0.0651151370369455 = (52.43178429744459 - 55.84588711706426) / 52.43178429744459 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.OneBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 164.25075566395796 > 92.2851495095848. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.35228933522408 (T) = (0 -163.8747689710087) / Math.Sqrt((143.02329389589656 / (44)) + (16.22922940493473 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8149749862788909 = (90.2903732612806 - 163.8747689710087) / 90.2903732612806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 310.76787154420674 > 274.3509429450091. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -111.29863505566097 (T) = (0 -311.0895533774449) / Math.Sqrt((6.253154581721741 / (44)) + (0.897096148235714 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19156472757473894 = (261.0765040105069 - 311.0895533774449) / 261.0765040105069 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.MaxBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 348.0319001373123 > 309.79362048353414. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.83165973434921 (T) = (0 -353.5951874503536) / Math.Sqrt((13.565760868429287 / (44)) + (147.07690069906315 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19100694387404754 = (296.88759521434605 - 353.5951874503536) / 296.88759521434605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 93.52595386406956 > 87.49037421456484. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.67919387519026 (T) = (0 -94.56537007215124) / Math.Sqrt((1.994177849795456 / (44)) + (2.0262244591457406 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13037513176501353 = (83.65839570841676 - 94.56537007215124) / 83.65839570841676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 332.2919695177382 > 256.9271980106822. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -99.32830626270359 (T) = (0 -331.01294301055424) / Math.Sqrt((27.255850608824886 / (43)) + (1.2362055137166439 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.340412095646524 = (246.9486392174759 - 331.01294301055424) / 246.9486392174759 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.MinBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 350.3191739402246 > 317.7074023347035. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.79798547552284 (T) = (0 -353.7052532391533) / Math.Sqrt((539.7915824313312 / (44)) + (4.328049285471413 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14707027257768623 = (308.35534813774746 - 353.7052532391533) / 308.35534813774746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 502.72571468925173 > 204.0375675266168. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -236.03164533572775 (T) = (0 -507.9521677742343) / Math.Sqrt((19.079789557955802 / (44)) + (19.787367787946028 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.5987139703715731 = (195.4628995593561 - 507.9521677742343) / 195.4628995593561 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 380.1969795053523 > 56.44663898218473. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -634.3310920483059 (T) = (0 -379.9802504809799) / Math.Sqrt((0.7174910983073881 / (44)) + (3.733666764546096 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -6.128521885163542 = (53.30421321590184 - 379.9802504809799) / 53.30421321590184 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.6089392454977 > 53.87802844270605. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 10/10/2022 3:25:05 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.278352210939316 (T) = (0 -55.435011729272496) / Math.Sqrt((0.86143850694881 / (44)) + (0.5799370755246951 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0712519918444626 = (51.747872723975505 - 55.435011729272496) / 51.747872723975505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.EqualsStaticBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 317.309277861128 > 279.8327724900773. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.965210194553883 (T) = (0 -319.07536600500964) / Math.Sqrt((16.495120499267266 / (44)) + (61.7921613966467 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1893845274600285 = (268.2693095784641 - 319.07536600500964) / 268.2693095784641 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 448.2456439459534 > 216.2684871690139. IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -144.52483333907728 (T) = (0 -443.6752252001872) / Math.Sqrt((36.53994756797863 / (44)) + (26.971138933989266 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1191888176217244 = (209.36087502486214 - 443.6752252001872) / 209.36087502486214 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 315.25478196933136 > 259.54115820613924. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -88.26406780252319 (T) = (0 -314.70891720772886) / Math.Sqrt((22.951536126677613 / (44)) + (0.9412116247602643 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27290864618530664 = (247.23605904544573 - 314.70891720772886) / 247.23605904544573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 345.1111401921121 > 258.165813162965. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -46.070707946216544 (T) = (0 -332.0592092689246) / Math.Sqrt((14.486053717725877 / (44)) + (47.650150782486485 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3509537886988026 = (245.79612718563368 - 332.0592092689246) / 245.79612718563368 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.GreaterThanBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 316.5320049738163 > 280.1337541248207. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.388063121714256 (T) = (0 -327.98114304767347) / Math.Sqrt((29.47096603115125 / (44)) + (1210.6521422210442 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.22222479887904842 = (268.34764222463673 - 327.98114304767347) / 268.34764222463673 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.MultiplyOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 318.72406267550923 > 258.63260132965286. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.79545681764409 (T) = (0 -316.0083802858664) / Math.Sqrt((5.416599416547792 / (44)) + (27.41363859392084 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2821857938421019 = (246.46067816657003 - 316.0083802858664) / 246.46067816657003 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 505.41885986509334 > 365.71550608370177. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 11/2/2022 9:35:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.30061275659514 (T) = (0 -500.5062347863658) / Math.Sqrt((56.90881752297864 / (44)) + (55.026380260453756 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4185846047727263 = (352.82085615651573 - 500.5062347863658) / 352.82085615651573 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_VectorOf<UInt16>.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 314.39431218294163 > 259.98935801989296. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -66.10001738708121 (T) = (0 -315.50286866045656) / Math.Sqrt((17.20380190343346 / (43)) + (10.140830718104137 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2776889135254393 = (246.9324616662057 - 315.50286866045656) / 246.9324616662057 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 34.49 μs 39.03 μs 1.13 0.04 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToWriter(Mode%3a%20Reflection).html>) 30.79 μs 33.59 μs 1.09 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToWriter(Mode%3a%20SourceGen).html>) 22.30 μs 25.58 μs 1.15 0.04 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToString(Mode%3a%20Reflection).html>) 36.79 μs 41.51 μs 1.13 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 40.03 μs 43.74 μs 1.09 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeObjectProperty(Mode%3a%20Reflection).html>) 45.70 μs 49.30 μs 1.08 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 26.59 μs 29.48 μs 1.11 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Location).SerializeToString(Mode%3a%20SourceGen).html>) 28.83 μs 32.12 μs 1.11 0.01 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 39.027864609374994 > 36.257634241487516. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.60726562958511 (T) = (0 -38262.05976176663) / Math.Sqrt((182275.91400652795 / (44)) + (173978.152838724 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10752298609671784 = (34547.41819545881 - 38262.05976176663) / 34547.41819545881 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.594423327817125 > 32.12156792155321. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.583780161765155 (T) = (0 -33700.08795683943) / Math.Sqrt((163720.16929902064 / (44)) + (110316.63236924146 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10942312611803429 = (30376.226313903244 - 33700.08795683943) / 30376.226313903244 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.582159727933114 > 23.472723592304447. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.08115761561713 (T) = (0 -25361.644691043435) / Math.Sqrt((105801.13594440717 / (43)) + (59475.28124031603 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.13189530337839125 = (22406.35208516725 - 25361.644691043435) / 22406.35208516725 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.51176486185383 > 38.82283285359894. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.33261124043491 (T) = (0 -41119.8463725658) / Math.Sqrt((169274.44991572478 / (44)) + (73507.4332630786 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10933845955973061 = (37066.9979195396 - 41119.8463725658) / 37066.9979195396 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 43.735452985491065 > 42.18598989657491. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.639254819434896 (T) = (0 -43779.79984112959) / Math.Sqrt((175147.40782908892 / (43)) + (323274.9988470072 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.09461313531277708 = (39995.682884455666 - 43779.79984112959) / 39995.682884455666 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 49.299854431216936 > 48.01024279370221. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.91748549237534 (T) = (0 -49619.86400372532) / Math.Sqrt((207877.373267261 / (43)) + (225443.89485067446 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.09286892279031336 = (45403.307724256505 - 49619.86400372532) / 45403.307724256505 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.481897202832748 > 27.658290878267106. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.305703601761074 (T) = (0 -29510.52584829897) / Math.Sqrt((99536.8190491016 / (44)) + (77542.77676724918 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12230803727200873 = (26294.497471505354 - 29510.52584829897) / 26294.497471505354 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<Location>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.11950734086242 > 30.419792036099206. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.27159413514304 (T) = (0 -32110.846038124026) / Math.Sqrt((74515.43776832985 / (44)) + (37302.23762755349 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11013543631288526 = (28925.160829722194 - 32110.846038124026) / 28925.160829722194 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(LoginViewModel).SystemTextJsonSourceGen.html>) 17.14 μs 18.51 μs 1.08 0.04 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(LoginViewModel).JsonNet.html>) 16.70 μs 18.92 μs 1.13 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(LoginViewModel).SystemTextJsonReflection.html>) 17.63 μs 19.03 μs 1.08 0.01 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToStream(LoginViewModel).DataContractJsonSerializer.html>) 33.23 μs 37.56 μs 1.13 0.01 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.505295857988166 > 18.15508380884888. IsChangePoint: Marked as a change because one of 12/5/2022 3:14:07 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.999572196045616 (T) = (0 -18654.758837174177) / Math.Sqrt((8983152.058799881 / (44)) + (54475.151798946375 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32336022061593755 = (14096.50868037397 - 18654.758837174177) / 14096.50868037397 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.922586732385266 > 17.75366461409213. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.731496716264736 (T) = (0 -18592.716242942963) / Math.Sqrt((53635.11299270575 / (44)) + (25594.43485267707 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09817373406346983 = (16930.57816466441 - 18592.716242942963) / 16930.57816466441 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.032808099261622 > 18.61170578237186. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.437285156756943 (T) = (0 -19023.273243237618) / Math.Sqrt((52561.2354663283 / (44)) + (40492.273804731536 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08714804151851938 = (17498.32820989685 - 19023.273243237618) / 17498.32820989685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToStream<LoginViewModel>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.55581462098906 > 35.911685872662105. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.62690083895573 (T) = (0 -38106.47584249063) / Math.Sqrt((179587.00457357688 / (44)) + (44974.10165331932 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12866241950556626 = (33762.50966093472 - 38106.47584249063) / 33762.50966093472 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).List(Size%3a%20512).html>) 383.79 μs 617.91 μs 1.61 0.06 False
[Array_Comparison - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).Array_Comparison(Size%3a%20512).html>) 408.07 μs 516.81 μs 1.27 0.02 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).Array(Size%3a%20512).html>) 377.62 μs 578.98 μs 1.53 0.03 False
[Array_ComparerStruct - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).Array_ComparerStruct(Size%3a%20512).html>) 429.30 μs 540.35 μs 1.26 0.03 False
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.Sort(IntStruct).Array_ComparerClass(Size%3a%20512).html>) 392.72 μs 505.37 μs 1.29 0.03 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<IntStruct>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 617.9100571857143 > 402.70034254500007. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.4859116437427 (T) = (0 -588240.7846526203) / Math.Sqrt((1921837831.1265845 / (44)) + (78135319.3119122 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.45946256125381574 = (403053.0143556859 - 588240.7846526203) / 403053.0143556859 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_Comparison(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 516.8099999571428 > 421.70759256. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.50545387315836 (T) = (0 -511944.2407898754) / Math.Sqrt((2056621817.0092337 / (44)) + (40340161.832411766 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22770923274712554 = (416991.4399392009 - 511944.2407898754) / 416991.4399392009 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 578.9832000000001 > 397.10661748499996. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.842099410893695 (T) = (0 -587748.0003667106) / Math.Sqrt((1873364097.3059938 / (44)) + (28694354.151973665 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4619137832338318 = (402040.12514785957 - 587748.0003667106) / 402040.12514785957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_ComparerStruct(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 540.3549333333332 > 452.535671035. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.809315312202946 (T) = (0 -539664.9143670598) / Math.Sqrt((1903328214.596936 / (44)) + (7192736.758631818 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2217212361833222 = (441725.08292724955 - 539664.9143670598) / 441725.08292724955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.Sort<IntStruct>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 505.3716533733334 > 417.14872600750004. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.246905471113953 (T) = (0 -515058.14839225396) / Math.Sqrt((2012353438.458307 / (44)) + (101567204.23979697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.22905244244660997 = (419069.30136109964 - 515058.14839225396) / 419069.30136109964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).OnesComplementOperatorBenchmark.html>) 103.30 ns 152.47 ns 1.48 0.06 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GetHashCodeBenchmark.html>) 389.47 ns 439.76 ns 1.13 0.10 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).AndNotBenchmark.html>) 133.06 ns 265.29 ns 1.99 0.06 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GreaterThanAllBenchmark.html>) 160.32 ns 276.67 ns 1.73 0.05 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).LessThanOrEqualAnyBenchmark.html>) 179.51 ns 218.61 ns 1.22 0.21 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).LessThanOrEqualAllBenchmark.html>) 72.93 ns 303.62 ns 4.16 0.06 False
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).EqualsAnyBenchmark.html>) 161.18 ns 214.30 ns 1.33 0.21 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).LessThanAllBenchmark.html>) 70.92 ns 259.77 ns 3.66 0.07 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).LessThanAnyBenchmark.html>) 173.28 ns 214.62 ns 1.24 0.17 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GreaterThanOrEqualAllBenchmark.html>) 169.75 ns 279.60 ns 1.65 0.13 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GreaterThanOrEqualAnyBenchmark.html>) 72.87 ns 193.11 ns 2.65 0.06 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).SumBenchmark.html>) 43.92 ns 58.81 ns 1.34 0.11 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).OnesComplementBenchmark.html>) 103.77 ns 154.31 ns 1.49 0.14 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).AllBitsSetBenchmark.html>) 44.43 ns 88.64 ns 2.00 0.06 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).ConditionalSelectBenchmark.html>) 179.06 ns 423.10 ns 2.36 0.09 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GreaterThanAnyBenchmark.html>) 71.20 ns 207.83 ns 2.92 0.08 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.47264424011468 > 110.82519017969018. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.867956137234592 (T) = (0 -153.25453940115202) / Math.Sqrt((924.289823604117 / (44)) + (1.340850038846517 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2591802388028533 = (121.7097717058016 - 153.25453940115202) / 121.7097717058016 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 439.7580900426385 > 402.1502746031667. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.01972953439485 (T) = (0 -441.23574793057713) / Math.Sqrt((211.36300126220962 / (43)) + (67.33255897788442 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15232486360556294 = (382.9091620482561 - 441.23574793057713) / 382.9091620482561 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 265.2922985001142 > 139.9171333982435. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.956821409640163 (T) = (0 -266.3603960995837) / Math.Sqrt((898.8202530967973 / (44)) + (12.949987955327273 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7615959345918584 = (151.204025207572 - 266.3603960995837) / 151.204025207572 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 276.6748982244662 > 169.50771925180308. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -176.28258088757852 (T) = (0 -276.0277035671172) / Math.Sqrt((7.471955570940214 / (44)) + (3.892039451990147 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7195048703991715 = (160.52743340182528 - 276.0277035671172) / 160.52743340182528 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 218.61124006524554 > 171.67319681379067. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.76623006233261 (T) = (0 -220.36018404981223) / Math.Sqrt((19.572271326813908 / (44)) + (34.79454584322473 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3284243917043036 = (165.88086264141904 - 220.36018404981223) / 165.88086264141904 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 303.6204846236748 > 76.17699797993951. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -69.77862395806059 (T) = (0 -266.57381941343715) / Math.Sqrt((8.129707707679321 / (44)) + (112.00366241313122 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.6241659073302994 = (73.55452985037479 - 266.57381941343715) / 73.55452985037479 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.29653743758996 > 169.05224266096252. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.29653992014761 (T) = (0 -213.08860588437759) / Math.Sqrt((19.150299782783385 / (44)) + (1.8552964525813493 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3090826247041026 = (162.7770484942025 - 213.08860588437759) / 162.7770484942025 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 259.77071250006713 > 74.41975955212955. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -122.74763679155394 (T) = (0 -259.96278931828033) / Math.Sqrt((1.1074249026625138 / (43)) + (34.951385885496634 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -2.6327793465604103 = (71.56030259983129 - 259.96278931828033) / 71.56030259983129 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 214.62088635903325 > 174.56417397916331. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.778377344403964 (T) = (0 -215.7220176799641) / Math.Sqrt((21.293861548072304 / (44)) + (23.409261358984452 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3223673757637079 = (163.13319704774023 - 215.7220176799641) / 163.13319704774023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.5989467061989 > 170.78810933931345. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -58.22935100424288 (T) = (0 -284.61169956388045) / Math.Sqrt((26.83207072155642 / (44)) + (53.655487339207774 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7200826170347411 = (165.46397059376366 - 284.61169956388045) / 165.46397059376366 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 193.11353402134753 > 77.69385057210354. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -135.86001570424625 (T) = (0 -193.9000738794162) / Math.Sqrt((9.502593113812498 / (43)) + (7.82397391435522 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -1.6229787309553587 = (73.92361653225935 - 193.9000738794162) / 73.92361653225935 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.81394773322155 > 46.24550068897012. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.585058006492986 (T) = (0 -59.130391718215726) / Math.Sqrt((0.8861725927413633 / (44)) + (0.9390637493295207 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3291690962359042 = (44.48673376899001 - 59.130391718215726) / 44.48673376899001 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 154.31126440729716 > 108.777183064458. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.83579877766591 (T) = (0 -154.34018498588426) / Math.Sqrt((885.3570474465772 / (44)) + (11.486694101125725 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25379300404372146 = (123.0986171466165 - 154.34018498588426) / 123.0986171466165 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 88.64495979258469 > 46.57683677952864. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.523626358069508 (T) = (0 -87.90658811267036) / Math.Sqrt((926.1179529203063 / (44)) + (0.6139266276166085 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4055938576882 = (62.540532339300036 - 87.90658811267036) / 62.540532339300036 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 423.09518901450116 > 191.0951660970058. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.67941344001706 (T) = (0 -421.2921423988013) / Math.Sqrt((946.9889769605157 / (44)) + (32.606702485849844 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1177884066439745 = (198.93023357626942 - 421.2921423988013) / 198.93023357626942 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 207.82518334870846 > 75.10623669303963. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -93.04141018067125 (T) = (0 -193.42164825454694) / Math.Sqrt((2.4517147189535518 / (44)) + (24.682105090953463 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.6838814540131166 = (72.06788063061806 - 193.42164825454694) / 72.06788063061806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(String).ReadOnlySpan(Size%3a%20512).html>) 4.78 μs 7.38 μs 1.54 0.07 False
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(String).Array(Size%3a%20512).html>) 3.94 μs 4.87 μs 1.24 0.08 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(String).Span(Size%3a%20512).html>) 4.80 μs 7.38 μs 1.54 0.08 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(String).ImmutableArray(Size%3a%20512).html>) 5.63 μs 8.25 μs 1.46 0.05 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<String>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.380461410011745 > 5.024140697670854. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.81318599499378 (T) = (0 -7421.459112021865) / Math.Sqrt((9980.994396855373 / (43)) + (23388.864892156966 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.5444515306679193 = (4805.239248144196 - 7421.459112021865) / 4805.239248144196 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.869565707077533 > 4.135200764450424. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -96.01989107059872 (T) = (0 -4871.599777168401) / Math.Sqrt((3129.673929449625 / (44)) + (310.11526457410264 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2328187767407558 = (3951.594402258872 - 4871.599777168401) / 3951.594402258872 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.384528346831825 > 5.037187653839056. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.115300114662912 (T) = (0 -7616.978159896628) / Math.Sqrt((25654.55764844882 / (44)) + (669693.7075912782 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5777577993866918 = (4827.7233443926 - 7616.978159896628) / 4827.7233443926 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<String>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.246971924915233 > 5.914429967667769. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -1736.0533764152974 (T) = (0 -8239.23475649095) / Math.Sqrt((19.184979690858164 / (43)) + (28.8777290407853 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.4622665300935515 = (5634.564278759652 - 8239.23475649095) / 5634.564278759652 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[EqualsAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).EqualsAnyBenchmark.html>) 174.23 ns 215.28 ns 1.24 0.20 False
[GreaterThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GreaterThanOrEqualAllBenchmark.html>) 170.06 ns 280.26 ns 1.65 0.07 False
[GreaterThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GreaterThanOrEqualAnyBenchmark.html>) 73.66 ns 191.98 ns 2.61 0.06 False
[OnesComplementOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).OnesComplementOperatorBenchmark.html>) 102.37 ns 174.58 ns 1.71 0.13 False
[LessThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).LessThanAllBenchmark.html>) 74.54 ns 258.22 ns 3.46 0.04 False
[ConditionalSelectBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).ConditionalSelectBenchmark.html>) 180.79 ns 420.62 ns 2.33 0.09 False
[LessThanOrEqualAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).LessThanOrEqualAllBenchmark.html>) 74.21 ns 260.58 ns 3.51 0.15 False
[AllBitsSetBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).AllBitsSetBenchmark.html>) 43.94 ns 89.62 ns 2.04 0.10 False
[SumBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).SumBenchmark.html>) 42.89 ns 57.41 ns 1.34 0.14 False
[OnesComplementBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).OnesComplementBenchmark.html>) 101.62 ns 152.03 ns 1.50 0.14 False
[GreaterThanAllBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GreaterThanAllBenchmark.html>) 159.32 ns 279.61 ns 1.76 0.11 False
[LessThanOrEqualAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).LessThanOrEqualAnyBenchmark.html>) 163.65 ns 221.49 ns 1.35 0.24 False
[GreaterThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GreaterThanAnyBenchmark.html>) 71.76 ns 189.88 ns 2.65 0.07 False
[LessThanAnyBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).LessThanAnyBenchmark.html>) 178.44 ns 220.48 ns 1.24 0.19 False
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GetHashCodeBenchmark.html>) 402.48 ns 445.75 ns 1.11 0.06 False
[AndNotBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).AndNotBenchmark.html>) 133.30 ns 265.31 ns 1.99 0.06 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.EqualsAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 215.27552334903794 > 176.4562089752584. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.69334345132834 (T) = (0 -215.4301093959571) / Math.Sqrt((27.085822312442588 / (44)) + (21.516185661990306 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32252327266505093 = (162.89324645443727 - 215.4301093959571) / 162.89324645443727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 280.2567650954564 > 171.81430909285712. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -138.24149834746305 (T) = (0 -282.89270759213235) / Math.Sqrt((7.245239538751324 / (44)) + (8.58888190155572 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7229225959940673 = (164.1935094762124 - 282.89270759213235) / 164.1935094762124 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 191.97781836367477 > 76.63388621841624. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -248.4504646338014 (T) = (0 -193.56093437031717) / Math.Sqrt((1.0849322903813299 / (43)) + (2.9293339458377736 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -1.642245688064864 = (73.25622111699913 - 193.56093437031717) / 73.25622111699913 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 174.5810953141518 > 107.77463127393676. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.713478775787123 (T) = (0 -156.41211077035263) / Math.Sqrt((1007.0796068148023 / (44)) + (42.63613913313348 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2783177205950168 = (122.35777401063305 - 156.41211077035263) / 122.35777401063305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 258.2169422132236 > 76.17285024259814. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -149.9250219600796 (T) = (0 -260.0573362976031) / Math.Sqrt((1.4431298724113473 / (43)) + (23.1450549886117 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -2.6214902767180965 = (71.80948074594167 - 260.0573362976031) / 71.80948074594167 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.ConditionalSelectBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 420.62118162129605 > 192.34481138132492. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.80060430174073 (T) = (0 -421.69937737977693) / Math.Sqrt((889.4701479858036 / (44)) + (19.168790864683192 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.1075231678823874 = (200.09240411031644 - 421.69937737977693) / 200.09240411031644 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 260.5776218243767 > 77.25182423784189. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -108.41134415635852 (T) = (0 -265.17715456161335) / Math.Sqrt((7.670170891726466 / (44)) + (44.174955818255356 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -2.5977963756889615 = (73.70543712631128 - 265.17715456161335) / 73.70543712631128 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AllBitsSetBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.62290117706213 > 46.37247324666787. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.5961796626988916 (T) = (0 -88.10984038888327) / Math.Sqrt((899.8310166381274 / (44)) + (1.2581207162564303 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4041252574630446 = (62.75069828747258 - 88.10984038888327) / 62.75069828747258 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.SumBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.40788461419556 > 46.096910393086. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.105713951986314 (T) = (0 -58.80405138969138) / Math.Sqrt((22.64060516070653 / (44)) + (0.6730269072788009 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3003081523442623 = (45.22316597313984 - 58.80405138969138) / 45.22316597313984 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.OnesComplementBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 152.03198907670367 > 112.02862839661704. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.833499147134994 (T) = (0 -153.69283294642875) / Math.Sqrt((922.0279507159312 / (44)) + (8.56657957010284 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25990292411264027 = (121.98783732062202 - 153.69283294642875) / 121.98783732062202 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAllBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 279.6143701981733 > 169.98023251101392. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -183.74360683515775 (T) = (0 -277.0701449832742) / Math.Sqrt((3.5364930067080502 / (44)) + (4.788942625039513 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7218616636441106 = (160.9131272467551 - 277.0701449832742) / 160.9131272467551 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanOrEqualAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 221.4864854810529 > 174.2258017454411. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.55343809999222 (T) = (0 -221.96659862514198) / Math.Sqrt((40.21916536225303 / (44)) + (104.42904424773032 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.32841436743467506 = (167.09138659331552 - 221.96659862514198) / 167.09138659331552 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GreaterThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 189.8781741387756 > 75.38388973506717. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -111.57651877306816 (T) = (0 -193.35019406958605) / Math.Sqrt((7.158351311759716 / (44)) + (15.290908708179412 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -1.6839244097019912 = (72.0401041738186 - 193.35019406958605) / 72.0401041738186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.LessThanAnyBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 220.48397331170403 > 177.86153820344765. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.762520084092696 (T) = (0 -216.66733899846) / Math.Sqrt((24.072429938189238 / (44)) + (44.65636488836184 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3319281072871707 = (162.67194739193636 - 216.66733899846) / 162.67194739193636 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 445.7480421242628 > 399.4597237807702. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.55596576236998 (T) = (0 -441.0319552833949) / Math.Sqrt((43.78276720430012 / (43)) + (55.85431143928741 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.15748269178193197 = (381.02682520844525 - 441.0319552833949) / 381.02682520844525 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.AndNotBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 265.3050836371302 > 141.74188462028792. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.574507665706363 (T) = (0 -265.80957018529807) / Math.Sqrt((938.0241318706455 / (44)) + (5.666993013626042 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7563426744276662 = (151.34265884185533 - 265.80957018529807) / 151.34265884185533 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Xml.Linq.Perf_XElement

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateElementWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.CreateElementWithNamespace.html>) 1.99 μs 2.21 μs 1.11 0.04 False
[CreateElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.CreateElement.html>) 937.75 ns 1.07 μs 1.14 0.08 False
[GetAttribute - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.GetAttribute.html>) 757.81 ns 894.70 ns 1.18 0.10 False
[GetElementWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.GetElementWithNamespace.html>) 1.81 μs 2.12 μs 1.17 0.04 False
[CreateWithElements - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.CreateWithElements.html>) 1.38 μs 1.54 μs 1.11 0.06 False
[CreateElementsWithNamespace - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace.html>) 2.50 μs 2.70 μs 1.08 0.04 False
[GetElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElement.GetElement.html>) 892.66 ns 988.75 ns 1.11 0.14 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElement*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XElement.CreateElementWithNamespace ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.211040138077859 > 2.100389333577351. IsChangePoint: Marked as a change because one of 9/26/2022 2:03:14 AM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.860628423650613 (T) = (0 -2232.587251321768) / Math.Sqrt((946.4615177808117 / (44)) + (1299.3058957972057 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13095560531100775 = (1974.0715204358673 - 2232.587251321768) / 1974.0715204358673 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.CreateElement ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0734141232031857 > 1004.40164518076. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.772063141573716 (T) = (0 -1068.3699852060795) / Math.Sqrt((888.2887003155968 / (42)) + (395.0560859364204 / (16))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (16) - 2, .025) and -0.1110521767600329 = (961.5839899810826 - 1068.3699852060795) / 961.5839899810826 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.GetAttribute ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 894.6969797958027 > 802.3248422744706. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.847294275228524 (T) = (0 -890.045071404894) / Math.Sqrt((337.9398602425005 / (44)) + (148.31703431120766 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15093345575909453 = (773.324527974441 - 890.045071404894) / 773.324527974441 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.GetElementWithNamespace ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.117267309720312 > 1.9081970583955528. IsChangePoint: Marked as a change because one of 9/26/2022 2:03:14 AM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.07679534027339 (T) = (0 -2081.7711942117976) / Math.Sqrt((805.9845049468938 / (44)) + (517.3293686310793 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14407664297020212 = (1819.6081591240195 - 2081.7711942117976) / 1819.6081591240195 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.CreateWithElements ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.540174590007358 > 1.4781743473704605. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.965635668222093 (T) = (0 -1541.059358633767) / Math.Sqrt((1212.2047524550048 / (44)) + (645.8762268704138 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0953566440120401 = (1406.9019136901568 - 1541.059358633767) / 1406.9019136901568 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.CreateElementsWithNamespace ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.696905256598663 > 2.5990119382469636. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.02406937796637 (T) = (0 -2726.4208866372037) / Math.Sqrt((1292.2093873568397 / (44)) + (426.5443243120044 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10824034681179182 = (2460.1350189790746 - 2726.4208866372037) / 2460.1350189790746 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Xml.Linq.Perf_XElement.GetElement ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 988.7495795416246 > 938.601110804423. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.929182216954846 (T) = (0 -1018.698787143382) / Math.Sqrt((447.46625130569197 / (44)) + (835.3353395897611 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14534136995008057 = (889.428090061727 - 1018.698787143382) / 889.428090061727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(CollectionsOfPrimitives).SystemTextJsonSourceGen.html>) 7.93 ms 9.24 ms 1.16 0.05 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 7.91 ms 9.06 ms 1.15 0.06 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(CollectionsOfPrimitives).JsonNet.html>) 11.60 ms 13.04 ms 1.12 0.02 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.24227435897436 > 8.462063765818433. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.007419013061448 (T) = (0 -9235632.204655094) / Math.Sqrt((19773048458.724934 / (44)) + (31899580957.25344 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15198503099879204 = (8017146.018510008 - 9235632.204655094) / 8017146.018510008 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.05979486923077 > 8.424784903258622. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.88525819814158 (T) = (0 -9080543.51337452) / Math.Sqrt((13331716923.87944 / (44)) + (12651601740.732069 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1397579460697713 = (7967080.681198117 - 9080543.51337452) / 7967080.681198117 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.041803688888889 > 12.26671775175. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.84208094392749 (T) = (0 -13083756.920062728) / Math.Sqrt((27593964456.268696 / (44)) + (33112952158.425175 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12777391257083323 = (11601400.57703362 - 13083756.920062728) / 11601400.57703362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Threading.Tests.Perf_Volatile

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Write_double - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_Volatile.Write_double.html>) 11.98 ns 17.47 ns 1.46 0.32 False
[Read_double - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_Volatile.Read_double.html>) 160.39 ns 204.97 ns 1.28 0.16 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_Volatile*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tests.Perf_Volatile.Write_double ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.469086319735197 > 12.461158222821243. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.87698401987291 (T) = (0 -17.02299173055952) / Math.Sqrt((0.13958116172686041 / (44)) + (0.16282924659904913 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.42514741429680625 = (11.944723443896487 - 17.02299173055952) / 11.944723443896487 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tests.Perf_Volatile.Read_double ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 204.96511242064193 > 169.08648846445166. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -57.3461197406137 (T) = (0 -206.8014533903788) / Math.Sqrt((4.499052468850505 / (44)) + (7.8513450793164585 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2809729673819847 = (161.44091925142934 - 206.8014533903788) / 161.44091925142934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonToString(IndexViewModel).JsonNet.html>) 944.80 μs 1.06 ms 1.13 0.01 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(IndexViewModel).SystemTextJsonSourceGen.html>) 861.03 μs 973.63 μs 1.13 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(IndexViewModel).SystemTextJsonReflection.html>) 1.06 ms 1.19 ms 1.12 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0629829053724054 > 1016.3299584202666. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/19/2022 9:06:05 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.243850100245517 (T) = (0 -1066962.2553938923) / Math.Sqrt((427730925.94666994 / (44)) + (216697321.60307646 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10286916698179721 = (967442.27451188 - 1066962.2553938923) / 967442.27451188 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 973.6281544542414 > 903.8397670545178. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.23712769976203 (T) = (0 -974408.4579733907) / Math.Sqrt((89592244.23725091 / (44)) + (105433997.90958825 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14198151003598983 = (853261.1512621443 - 974408.4579733907) / 853261.1512621443 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1899350477941173 > 1.1114953221320005. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.12292315613 (T) = (0 -1192514.5735782231) / Math.Sqrt((1460299571.1467059 / (44)) + (44096215.74790041 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11915269779941293 = (1065551.2656343156 - 1192514.5735782231) / 1065551.2656343156 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromReader(Mode%3a%20SourceGen).html>) 22.01 μs 25.01 μs 1.14 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 14.06 μs 16.31 μs 1.16 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromReader(Mode%3a%20Reflection).html>) 21.82 μs 24.72 μs 1.13 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromString(Mode%3a%20SourceGen).html>) 19.51 μs 21.83 μs 1.12 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LoginViewModel).DeserializeFromString(Mode%3a%20Reflection).html>) 19.27 μs 21.81 μs 1.13 0.01 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 25.010078397024326 > 23.551470893533107. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.24087033757039 (T) = (0 -25212.56875847837) / Math.Sqrt((45886.02287946963 / (44)) + (60398.34349327439 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13544430613880304 = (22205.024607694184 - 25212.56875847837) / 22205.024607694184 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.312085501745198 > 14.706930784244822. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.48397582368739 (T) = (0 -16100.039674580627) / Math.Sqrt((35232.5486011138 / (44)) + (52268.69923167575 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1521786265284771 = (13973.562175068433 - 16100.039674580627) / 13973.562175068433 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.717540171496488 > 22.91900223339466. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -52.164979942043175 (T) = (0 -24787.406382664412) / Math.Sqrt((69337.87264348948 / (44)) + (25891.57845929233 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13756590926982584 = (21789.86393726831 - 24787.406382664412) / 21789.86393726831 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.827602171408536 > 20.583633238871922. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.70419335088268 (T) = (0 -21923.411374376137) / Math.Sqrt((58553.8431556404 / (44)) + (30398.846266768734 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11666422843577279 = (19632.948576750354 - 21923.411374376137) / 19632.948576750354 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LoginViewModel>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 21.81424923291492 > 20.163666514168547. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -50.639467077369446 (T) = (0 -21630.71394815409) / Math.Sqrt((29828.31898628843 / (44)) + (24244.566642510985 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12629561453302504 = (19205.183496272806 - 21630.71394815409) / 19205.183496272806 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>) 11.41 μs 12.18 μs 1.07 0.07 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20Reflection).html>) 19.02 μs 20.65 μs 1.09 0.02 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 27.61 μs 29.01 μs 1.05 0.02 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 14.49 μs 15.59 μs 1.08 0.04 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20Reflection).html>) 24.74 μs 26.60 μs 1.08 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20SourceGen).html>) 15.98 μs 17.84 μs 1.12 0.04 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.18253156987094 > 11.775935742178245. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.654879949833427 (T) = (0 -12303.78690100349) / Math.Sqrt((44009.52063099567 / (44)) + (113268.66110471333 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08705232073650836 = (11318.486393246767 - 12303.78690100349) / 11318.486393246767 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 20.646459993403692 > 19.978085230868466. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.972555199028488 (T) = (0 -20469.827686001645) / Math.Sqrt((76524.26199084289 / (44)) + (64294.42645586132 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08194461130898636 = (18919.47838368205 - 20469.827686001645) / 18919.47838368205 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.01452113650497 > 28.99069698825257. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.087628965938354 (T) = (0 -29155.382909606862) / Math.Sqrt((98567.11089986518 / (43)) + (77154.0043223023 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.05798001986866605 = (27557.593113361545 - 29155.382909606862) / 27557.593113361545 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.589593789702443 > 15.241038384586636. IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.836595685905245 (T) = (0 -15731.806894798236) / Math.Sqrt((63253.323149411204 / (43)) + (42186.48007712569 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.08311838878852086 = (14524.549723871298 - 15731.806894798236) / 14524.549723871298 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.59537824943117 > 26.036218026834895. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.105109953644213 (T) = (0 -26762.14350858813) / Math.Sqrt((122937.61836329159 / (44)) + (122196.44806495526 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07164287032984644 = (24973.005699511512 - 26762.14350858813) / 24973.005699511512 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.835673197161487 > 17.268990502621943. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.87981088284828 (T) = (0 -17668.188191334753) / Math.Sqrt((86111.5586924984 / (44)) + (24861.509831062973 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07648334375913421 = (16412.876514778713 - 17668.188191334753) / 16412.876514778713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_DateTimes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 156.32 ms 202.86 ms 1.30 0.05 False
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 143.63 ms 178.92 ms 1.25 0.03 False
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 139.40 ms 183.60 ms 1.32 0.07 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_DateTimes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 202.85985714285712 > 168.7526050525. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -35.36593360634111 (T) = (0 -204038708.37711847) / Math.Sqrt((28075835500217.63 / (44)) + (14811834692480.26 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28367985313305444 = (158948282.84414127 - 204038708.37711847) / 158948282.84414127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 178.91735707142857 > 151.96354384000006. IsChangePoint: Marked as a change because one of 10/23/2022 1:14:54 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.867882658734345 (T) = (0 -178382599.43177044) / Math.Sqrt((21713712937211.707 / (44)) + (5039605622221.208 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26367169551756103 = (141162138.91988015 - 178382599.43177044) / 141162138.91988015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 183.60126693333333 > 148.97672873125003. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.95131653315955 (T) = (0 -182436460.14859587) / Math.Sqrt((28518743329698.395 / (44)) + (17202591129047.545 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.27002209030469065 = (143648257.41324514 - 182436460.14859587) / 143648257.41324514 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 12.22 μs 13.50 μs 1.10 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 12.87 μs 14.04 μs 1.09 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 15.87 μs 17.24 μs 1.09 0.02 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 7.77 μs 8.68 μs 1.12 0.02 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToString(Mode%3a%20Reflection).html>) 17.94 μs 19.40 μs 1.08 0.01 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 25.15 μs 27.57 μs 1.10 0.01 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(LoginViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 10.95 μs 12.45 μs 1.14 0.04 False

graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.500065222270493 > 12.472141494677935. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.956033249986724 (T) = (0 -13298.38926432964) / Math.Sqrt((26712.236822339717 / (44)) + (23960.34020117536 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11387454599907446 = (11938.857308568653 - 13298.38926432964) / 11938.857308568653 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 14.042053989921781 > 13.51979464646334. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.74690800399611 (T) = (0 -14058.436855889016) / Math.Sqrt((21723.617688496783 / (44)) + (6964.777543548437 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0906393247949854 = (12890.087984432133 - 14058.436855889016) / 12890.087984432133 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.24222772718911 > 16.684138450944. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.138145975821878 (T) = (0 -17384.75095450249) / Math.Sqrt((40008.267710216314 / (44)) + (53280.53420638169 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1116348734839553 = (15638.903896580043 - 17384.75095450249) / 15638.903896580043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.675719825119971 > 8.091998583757455. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.10223059421111 (T) = (0 -8535.319366350517) / Math.Sqrt((13091.607143834828 / (44)) + (10940.176463376683 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11327901794123663 = (7666.8285567213 - 8535.319366350517) / 7666.8285567213 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.399361372225414 > 18.688308524856595. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.28257325292913 (T) = (0 -19376.945432438577) / Math.Sqrt((42313.459452339215 / (44)) + (25786.01946625983 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09440334555537375 = (17705.488119286965 - 19376.945432438577) / 17705.488119286965 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 27.572399408983458 > 26.310828861263236. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.263548522068064 (T) = (0 -27228.53825727965) / Math.Sqrt((59881.245648098105 / (44)) + (77205.73841361699 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09139160278989156 = (24948.458635448686 - 27228.53825727965) / 24948.458635448686 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.45346999 > 11.517736704047387. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.776822161882247 (T) = (0 -12168.44488227806) / Math.Sqrt((23591.656407228125 / (44)) + (39062.72111634103 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11146673950267212 = (10948.096285564832 - 12168.44488227806) / 10948.096285564832 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Hashtable>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 978.70 μs 1.10 ms 1.13 0.02 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromReader(Mode%3a%20Reflection).html>) 1.14 ms 1.26 ms 1.11 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.18 ms 1.28 ms 1.08 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20Reflection).html>) 1.05 ms 1.18 ms 1.13 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(Hashtable).DeserializeFromString(Mode%3a%20SourceGen).html>) 1.06 ms 1.22 ms 1.15 0.01 False

graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Hashtable&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1042867421016485 > 1025.9454395733173. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.63551332082927 (T) = (0 -1102883.571077043) / Math.Sqrt((198322190.88064268 / (44)) + (195161835.0498359 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1413601990628282 = (966288.7947053188 - 1102883.571077043) / 966288.7947053188 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2623761142751477 > 1.1875138666015623. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.764576430245334 (T) = (0 -1275307.8167710386) / Math.Sqrt((160690894.48067346 / (44)) + (114012059.72277685 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1368515891950715 = (1121789.1841748655 - 1275307.8167710386) / 1121789.1841748655 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2780762652529762 > 1.1917771490234375. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.3379004929101 (T) = (0 -1275696.408741361) / Math.Sqrt((329966677.80608654 / (44)) + (84451086.60162188 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12940016840087004 = (1129534.4594712018 - 1275696.408741361) / 1129534.4594712018 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1847908649839742 > 1.1055895636658657. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.06249447933119 (T) = (0 -1185567.6057828495) / Math.Sqrt((300621852.10290724 / (44)) + (78510740.90767102 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12924375306198127 = (1049877.4977219438 - 1185567.6057828495) / 1049877.4977219438 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<Hashtable>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2158099105029585 > 1.1167917783413464. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.69320327166195 (T) = (0 -1193486.9898286236) / Math.Sqrt((341332767.8964906 / (44)) + (251275677.0846967 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1333299263144281 = (1053079.9214927866 - 1193486.9898286236) / 1053079.9214927866 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.Trim(input%3a%20%22%20abcdefg%20%22).html>) 433.36 ns 577.04 ns 1.33 0.07 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAA5AAAA%22%2c%20value%3a%20%225%22%2c%20comparisonType%3a%20InvariantCulture).html>) 657.45 ns 873.49 ns 1.33 0.06 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22foobardzsdzs%22%2c%20value%3a%20%22rddzs%22%2c%20comparisonType%3a%20InvariantCulture).html>) 871.30 ns 1.31 μs 1.50 0.07 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22StrIng%22%2c%20value%3a%20%22string%22%2c%20comparisonType%3a%20OrdinalIgnoreCase).html>) 326.69 ns 391.13 ns 1.20 0.13 False
[Trim - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.Trim(input%3a%20%22abcdefg%22).html>) 270.30 ns 332.65 ns 1.23 0.13 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22Hello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello%20Worldbbbbbbbbbbbbbbbbbbb.html>) 259.30 ns 357.45 ns 1.38 0.12 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22string1%22%2c%20value%3a%20%22string2%22%2c%20comparisonType%3a%20InvariantCulture).html>) 688.14 ns 1.09 μs 1.58 0.05 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%22%2c%20value%3a%20%22x%22%2c%20comparisonType%3a%20InvariantCultureIgno.html>) 5.15 μs 7.59 μs 1.47 0.13 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22ABCDE%22%2c%20value%3a%20%22c%22%2c%20comparisonType%3a%20InvariantCultureIgnoreCase).html>) 544.73 ns 735.81 ns 1.35 0.10 False
[IndexOfString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.ReadOnlySpan.IndexOfString(input%3a%20%22More%20Test%27s%22%2c%20value%3a%20%22Tests%22%2c%20comparisonType%3a%20OrdinalIgnoreCase).html>) 403.52 ns 526.64 ns 1.31 0.12 False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlySpan.Trim(input: " abcdefg ") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 577.0380203285129 > 461.2558287077971. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/23/2022 1:14:54 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -88.02642422233137 (T) = (0 -577.8863261298513) / Math.Sqrt((68.15731362964844 / (44)) + (12.57516276099925 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.30780333041522323 = (441.8755578075905 - 577.8863261298513) / 441.8755578075905 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAA5AAAA", value: "5", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 873.4903386360205 > 649.3101542856575. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -63.241138633257094 (T) = (0 -854.9541052021013) / Math.Sqrt((260.038304406466 / (44)) + (102.46343811689381 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3587543240022752 = (629.2190501987096 - 854.9541052021013) / 629.2190501987096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "foobardzsdzs", value: "rddzs", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.306168882993538 > 938.8906679360794. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -86.17907292800993 (T) = (0 -1307.8359559415344) / Math.Sqrt((528.4260497864443 / (44)) + (177.17616244661355 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4740829807559636 = (887.2200364668944 - 1307.8359559415344) / 887.2200364668944 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 391.1277179877027 > 328.087038847824. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.96682765811876 (T) = (0 -393.0172416385522) / Math.Sqrt((45.41119034198536 / (44)) + (40.80405962010534 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.245311105283435 = (315.5976365834309 - 393.0172416385522) / 315.5976365834309 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.Trim(input: "abcdefg") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 332.64720172422346 > 287.07080088593005. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -60.996363639358954 (T) = (0 -336.1630069270175) / Math.Sqrt((13.782116958340016 / (44)) + (11.250342130559162 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23016262913521973 = (273.26712661018934 - 336.1630069270175) / 273.26712661018934 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 357.44943114725186 > 272.69300314940625. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.53658798135491 (T) = (0 -361.39343628323564) / Math.Sqrt((26.247564921322194 / (43)) + (69.45242929429894 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.3807966697435826 = (261.7282067680155 - 361.39343628323564) / 261.7282067680155 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "string1", value: "string2", comparisonType: InvariantCulture) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.085119457962551 > 722.8320356780538. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -76.44920558753981 (T) = (0 -1090.1331039827217) / Math.Sqrt((129.06656775273123 / (44)) + (357.5627038821388 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5694836950836027 = (694.5807130061665 - 1090.1331039827217) / 694.5807130061665 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.592649765826875 > 5.405777487356086. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.662042975834474 (T) = (0 -7699.938831976187) / Math.Sqrt((68190.92801857377 / (44)) + (45568.3972435807 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4760091893201839 = (5216.728247825207 - 7699.938831976187) / 5216.728247825207 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 735.8082722789264 > 571.9617020996915. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.70072220298237 (T) = (0 -753.2370935039677) / Math.Sqrt((77.04553999049918 / (44)) + (324.57492971776134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.37772444136631594 = (546.7255068487918 - 753.2370935039677) / 546.7255068487918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 526.6400773239342 > 426.3431634947769. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.030154978069085 (T) = (0 -513.211785584171) / Math.Sqrt((92.39118843420873 / (44)) + (64.47150465041425 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24578295427810015 = (411.95922919138377 - 513.211785584171) / 411.95922919138377 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Perf_Quaternion

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcatenateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark.html>) 222.89 ns 235.15 ns 1.06 0.12 False
[SubtractionOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.SubtractionOperatorBenchmark.html>) 174.52 ns 201.46 ns 1.15 0.07 False
[NegationOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark.html>) 104.23 ns 122.26 ns 1.17 0.10 False
[MultiplyByQuaternionBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionBenchmark.html>) 262.34 ns 284.04 ns 1.08 0.10 False
[AddOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.AddOperatorBenchmark.html>) 167.87 ns 204.69 ns 1.22 0.11 False
[NegateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.NegateBenchmark.html>) 117.04 ns 137.41 ns 1.17 0.13 False
[MultiplyByScalarOperatorBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarOperatorBenchmark.html>) 106.95 ns 125.45 ns 1.17 0.13 False
[MultiplyByScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark.html>) 120.62 ns 141.06 ns 1.17 0.13 False
[ConjugateBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark.html>) 102.39 ns 119.12 ns 1.16 0.07 False
[SubtractBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.SubtractBenchmark.html>) 216.28 ns 247.20 ns 1.14 0.11 False
[CreateFromVector3WithScalarBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.CreateFromVector3WithScalarBenchmark.html>) 77.80 ns 89.65 ns 1.15 0.11 False
[DivideBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.DivideBenchmark.html>) 208.01 ns 257.16 ns 1.24 0.07 False
[CreateFromYawPitchRollBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.CreateFromYawPitchRollBenchmark.html>) 138.17 ns 158.40 ns 1.15 0.38 False
[AddBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.AddBenchmark.html>) 206.26 ns 250.28 ns 1.21 0.06 False
[CreateFromAxisAngleBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Perf_Quaternion.CreateFromAxisAngleBenchmark.html>) 112.19 ns 137.02 ns 1.22 0.13 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Quaternion*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_Quaternion.ConcatenateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 235.1517529815805 > 230.5934505693082. IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.041764828864029 (T) = (0 -236.3591826010228) / Math.Sqrt((62.12952059200729 / (44)) + (6.920437579851805 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08169258844017284 = (218.5086457344212 - 236.3591826010228) / 218.5086457344212 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.SubtractionOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 201.45705828985027 > 177.28620368969692. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.380855997894535 (T) = (0 -203.46500840569743) / Math.Sqrt((5.810686517901265 / (44)) + (6.651970555131365 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21456946252030168 = (167.52027338436108 - 203.46500840569743) / 167.52027338436108 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.NegationOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 122.26321974105626 > 109.84888923577084. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -15.424951248710407 (T) = (0 -124.82341912151956) / Math.Sqrt((1.4241320122558279 / (44)) + (23.844126139426916 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.186774721240335 = (105.17869726030459 - 124.82341912151956) / 105.17869726030459 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByQuaternionBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 284.04396048515866 > 267.9220649442433. IsChangePoint: Marked as a change because one of 9/23/2022 4:51:48 AM, 10/11/2022 6:23:40 AM, 10/19/2022 1:55:59 PM, 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -5.154891205887934 (T) = (0 -283.88301528654114) / Math.Sqrt((176.63094332115233 / (44)) + (241.7664600077799 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09143171976163564 = (260.1014888485558 - 283.88301528654114) / 260.1014888485558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.AddOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 204.69257814865279 > 175.7869805688229. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.403935517614546 (T) = (0 -205.2394318075707) / Math.Sqrt((7.203057855465016 / (44)) + (35.040881224049414 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2199473505551309 = (168.23630275042404 - 205.2394318075707) / 168.23630275042404 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.NegateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 137.4084262157761 > 122.29742405565518. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.58250331368884 (T) = (0 -135.68853587571715) / Math.Sqrt((9.483543200863231 / (44)) + (1.8602955127060103 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15119350135979306 = (117.86770487797357 - 135.68853587571715) / 117.86770487797357 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarOperatorBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 125.44827424172803 > 112.37457322909118. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.739933587645902 (T) = (0 -127.35814783518194) / Math.Sqrt((9.117867304687104 / (44)) + (27.326002561179955 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18157482349008164 = (107.78678193142036 - 127.35814783518194) / 107.78678193142036 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.MultiplyByScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 141.0599191600149 > 125.16915979425016. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 4:51:05 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.336380892183973 (T) = (0 -141.62817549771478) / Math.Sqrt((13.955444319587114 / (44)) + (26.355966563283697 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17066816273107666 = (120.98063311750735 - 141.62817549771478) / 120.98063311750735 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.ConjugateBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 119.12301396537099 > 109.72106805035702. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.598088555725845 (T) = (0 -120.50767754607128) / Math.Sqrt((4.821417640203346 / (44)) + (1.015038143498032 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15586454140217562 = (104.25761257446638 - 120.50767754607128) / 104.25761257446638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.SubtractBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 247.19988483399052 > 217.19092212222475. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.23715011573097 (T) = (0 -249.7088548552342) / Math.Sqrt((24.339174557655927 / (44)) + (34.6218923943323 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.19314727505566437 = (209.28586107996134 - 249.7088548552342) / 209.28586107996134 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.CreateFromVector3WithScalarBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 89.64618859330011 > 82.69928740338084. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.546778865524104 (T) = (0 -92.3361714771041) / Math.Sqrt((1.6203316519442827 / (44)) + (1.2553831401820108 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.16907595503358558 = (78.98218338983068 - 92.3361714771041) / 78.98218338983068 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.DivideBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 257.15894515784817 > 217.5167983685166. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.77627828688584 (T) = (0 -250.627939175599) / Math.Sqrt((42.264297694439904 / (43)) + (62.841234004764445 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.1923315379363088 = (210.1998741133583 - 250.627939175599) / 210.1998741133583 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.CreateFromYawPitchRollBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 158.39781123420408 > 146.58166210896766. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 11/21/2022 12:41:05 AM, 12/8/2022 9:11:55 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.5473396380218905 (T) = (0 -164.4548061594726) / Math.Sqrt((90.55916414137143 / (36)) + (148.52209291893564 / (22))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (22) - 2, .025) and -0.1379060893816846 = (144.52405843863096 - 164.4548061594726) / 144.52405843863096 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.AddBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 250.28065559044128 > 221.09239670770873. IsChangePoint: Marked as a change because one of 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.45150397166456 (T) = (0 -250.1275604449211) / Math.Sqrt((35.58545537110203 / (44)) + (3.3200377667604597 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.19059158304196258 = (210.08678711287791 - 250.1275604449211) / 210.08678711287791 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Perf_Quaternion.CreateFromAxisAngleBenchmark ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 137.02391191207604 > 116.71726061773896. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.70034766789755 (T) = (0 -138.77532857718668) / Math.Sqrt((5.2855122017772525 / (44)) + (1.3199479216948367 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2544390050024724 = (110.62740238766186 - 138.77532857718668) / 110.62740238766186 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToString(value%3a%2012345).html>) 3.34 μs 4.01 μs 1.20 0.13 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.Parse(value%3a%20%22-1.7976931348623157e%2b308%22).html>) 2.70 μs 3.09 μs 1.15 0.04 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22E%22).html>) 3.59 μs 4.25 μs 1.18 0.08 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22G%22).html>) 6.14 μs 7.76 μs 1.26 0.03 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.TryParse(value%3a%20%221.7976931348623157e%2b308%22).html>) 2.67 μs 3.07 μs 1.15 0.05 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 3.46 μs 4.27 μs 1.24 0.08 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 4.60 μs 5.57 μs 1.21 0.05 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22G%22).html>) 6.04 μs 7.61 μs 1.26 0.02 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.Parse(value%3a%20%221.7976931348623157e%2b308%22).html>) 2.66 μs 3.01 μs 1.13 0.04 False
[IsNaN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.IsNaN(value%3a%20NaN).html>) 13.56 ms 15.19 ms 1.12 0.07 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22F50%22).html>) 8.55 μs 9.79 μs 1.15 0.01 False
[ToStringWithCultureInfo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithCultureInfo(value%3a%20-1.7976931348623157E%2b308%2c%20culture%3a%20zh).html>) 6.02 μs 7.61 μs 1.27 0.03 False
[ToStringWithCultureInfo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithCultureInfo(value%3a%2012345%2c%20culture%3a%20zh).html>) 3.39 μs 4.20 μs 1.24 0.07 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22G17%22).html>) 4.44 μs 5.53 μs 1.25 0.06 False
[ToStringWithCultureInfo - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithCultureInfo(value%3a%201.7976931348623157E%2b308%2c%20culture%3a%20zh).html>) 6.04 μs 7.43 μs 1.23 0.03 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22G%22).html>) 3.52 μs 4.20 μs 1.19 0.07 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%201.7976931348623157E%2b308%2c%20format%3a%20%22R%22).html>) 6.20 μs 7.78 μs 1.25 0.01 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22G17%22).html>) 6.25 μs 7.25 μs 1.16 0.03 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%2012345%2c%20format%3a%20%22R%22).html>) 3.50 μs 4.31 μs 1.23 0.08 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22E%22).html>) 3.58 μs 4.21 μs 1.18 0.09 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.Parse(value%3a%20%2212345%22).html>) 1.39 μs 1.48 μs 1.06 0.06 False
[IsNaN - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.IsNaN(value%3a%200).html>) 14.33 ms 15.45 ms 1.08 0.06 False
[ToStringWithFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToStringWithFormat(value%3a%20-1.7976931348623157E%2b308%2c%20format%3a%20%22R%22).html>) 6.16 μs 7.87 μs 1.28 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToString(value%3a%201.7976931348623157E%2b308).html>) 5.90 μs 7.52 μs 1.27 0.03 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.TryParse(value%3a%20%2212345%22).html>) 1.39 μs 1.54 μs 1.10 0.05 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Double.ToString(value%3a%20-1.7976931348623157E%2b308).html>) 5.98 μs 7.57 μs 1.27 0.05 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Double.ToString(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.007964429802115 > 3.567327812300854. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.22338745389139 (T) = (0 -4109.855819184752) / Math.Sqrt((12491.446312015627 / (44)) + (15190.136011390685 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20253462643212658 = (3417.661104178376 - 4109.855819184752) / 3417.661104178376 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0937871753673116 > 2.841944813272095. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.49368306178389 (T) = (0 -3110.257870023082) / Math.Sqrt((1988.7623146828034 / (44)) + (2390.4105728054246 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13280053570750366 = (2745.63594559084 - 3110.257870023082) / 2745.63594559084 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.251043961086638 > 3.7728727852179724. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.773390432811656 (T) = (0 -4329.826341319235) / Math.Sqrt((12681.401849220902 / (44)) + (11863.976340757134 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1978838114052131 = (3614.5628650244503 - 4329.826341319235) / 3614.5628650244503 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.761912902966979 > 6.479074947419734. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.06830037387533 (T) = (0 -7766.244019767718) / Math.Sqrt((32698.073508985544 / (43)) + (8459.687152056245 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.2576442913875937 = (6175.231003671957 - 7766.244019767718) / 6175.231003671957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0694124201095643 > 2.8065384644514464. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.817855053778143 (T) = (0 -3049.932871477536) / Math.Sqrt((4203.015906878571 / (44)) + (1957.8183711058277 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1267371521490177 = (2706.8716653750353 - 3049.932871477536) / 2706.8716653750353 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.270035436353529 > 3.751264061799725. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.955984284132565 (T) = (0 -4139.760351438985) / Math.Sqrt((10959.953263099422 / (44)) + (11171.501715397471 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18988971523190779 = (3479.112642495739 - 4139.760351438985) / 3479.112642495739 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.572757363816738 > 4.851877503583553. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.235214858151615 (T) = (0 -5634.169024183261) / Math.Sqrt((9725.058133246272 / (44)) + (17786.370409910756 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24165153709588982 = (4537.641081942419 - 5634.169024183261) / 4537.641081942419 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.6127263353909465 > 6.387530129958231. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -51.40878351909248 (T) = (0 -7666.265134977503) / Math.Sqrt((17490.916926749545 / (43)) + (7486.7439564504475 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.2528685079947591 = (6118.970255903002 - 7666.265134977503) / 6118.970255903002 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0059218596813726 > 2.865866743145198. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.156804270892902 (T) = (0 -3045.5932922243546) / Math.Sqrt((1290.9647049352304 / (44)) + (2397.597727305665 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.13248901563156032 = (2689.291684234045 - 3045.5932922243546) / 2689.291684234045 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.IsNaN(value: NaN) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.190846153846154 > 14.234513954335938. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -41.94035598368477 (T) = (0 -15194773.648595084) / Math.Sqrt((61284221369.99116 / (44)) + (202845576.88651237 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11546191440996922 = (13621956.475880628 - 15194773.648595084) / 13621956.475880628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "F50") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.789259735374578 > 9.146531132893886. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.313205033248916 (T) = (0 -9830.016071342898) / Math.Sqrt((35442.427705151356 / (44)) + (12624.648933281667 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13768912971686156 = (8640.335760076488 - 9830.016071342898) / 8640.335760076488 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.613013632067719 > 6.301469557385627. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.606534899950766 (T) = (0 -7517.418159136286) / Math.Sqrt((20069.93967193389 / (44)) + (10782.135690083154 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25532589125423766 = (5988.419590091768 - 7517.418159136286) / 5988.419590091768 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.204912150943577 > 3.5966998000459376. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.76859500411105 (T) = (0 -4113.967151393631) / Math.Sqrt((11647.735756093263 / (44)) + (8483.980696367202 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.20990019575062568 = (3400.253314978029 - 4113.967151393631) / 3400.253314978029 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.527874444043755 > 4.796128179118978. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.42475539059504 (T) = (0 -5522.799565786788) / Math.Sqrt((8662.526498462608 / (44)) + (5706.627273649851 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.23292895742492856 = (4479.414270001087 - 5522.799565786788) / 4479.414270001087 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.432772422161173 > 6.329103308598106. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.69551790322038 (T) = (0 -7478.928016191901) / Math.Sqrt((22520.32338638768 / (44)) + (9908.608189415181 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2645502206801136 = (5914.298929282149 - 7478.928016191901) / 5914.298929282149 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.202325768091321 > 3.6841785235500835. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.314790185029814 (T) = (0 -4290.591457483246) / Math.Sqrt((6672.218876748425 / (43)) + (6822.028308051369 / (15))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (15) - 2, .025) and -0.22851763905553726 = (3492.494792978127 - 4290.591457483246) / 3492.494792978127 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.783643840214837 > 6.503370342395788. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -61.31901204815349 (T) = (0 -7791.252408104788) / Math.Sqrt((14904.857400781637 / (44)) + (5500.155400321033 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26427376389012214 = (6162.6307771596885 - 7791.252408104788) / 6162.6307771596885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G17") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.254949862018034 > 6.669478764666795. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.8552807451931 (T) = (0 -7231.195867322884) / Math.Sqrt((9911.048957079212 / (44)) + (6207.116834102087 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13875884184502732 = (6350.067812081121 - 7231.195867322884) / 6350.067812081121 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.309053410467936 > 3.752600226027928. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.389844057248176 (T) = (0 -4305.776070391311) / Math.Sqrt((7405.668630496272 / (44)) + (8481.45843481206 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21743460639166126 = (3536.761685420743 - 4305.776070391311) / 3536.761685420743 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.210795258241005 > 3.759061023009282. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.43480926932122 (T) = (0 -4239.141466431937) / Math.Sqrt((6819.427712586825 / (44)) + (3386.816040843614 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2015119157817016 = (3528.1726387823287 - 4239.141466431937) / 3528.1726387823287 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.Parse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.4784119776018763 > 1.4427011492025617. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -23.288879705321637 (T) = (0 -1508.8676915529898) / Math.Sqrt((395.80297926637564 / (44)) + (315.5032750893407 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0923949167259142 = (1381.2474485649498 - 1508.8676915529898) / 1381.2474485649498 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.IsNaN(value: 0) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.447295816666667 > 14.9667117515625. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.968335215347615 (T) = (0 -15477089.52996566) / Math.Sqrt((55695560973.33096 / (44)) + (16099434039.558393 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11485528014285944 = (13882599.65722403 - 15477089.52996566) / 13882599.65722403 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.871234282115869 > 6.6208551943567375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.4168591092091 (T) = (0 -7823.478698605132) / Math.Sqrt((16300.742982517988 / (44)) + (7411.463082315735 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.25710084737537614 = (6223.429659553006 - 7823.478698605132) / 6223.429659553006 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.521179229368931 > 6.3427423749597915. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.05676141519469 (T) = (0 -7500.50401182481) / Math.Sqrt((228361.75863766184 / (44)) + (4042.6890849808356 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.24623432674591053 = (6018.534276302322 - 7500.50401182481) / 6018.534276302322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.TryParse(value: "12345") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.536340678887736 > 1.4495150376420447. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.880098615219328 (T) = (0 -1509.5148374217017) / Math.Sqrt((573.0145870599216 / (44)) + (588.5669183108964 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09364795951908853 = (1380.2566212307322 - 1509.5148374217017) / 1380.2566212307322 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.570442076072927 > 6.326565744442085. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.21446212775948 (T) = (0 -7598.26694443475) / Math.Sqrt((20223.780333121733 / (44)) + (8465.847538599515 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.26146068214961243 = (6023.387848669846 - 7598.26694443475) / 6023.387848669846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Threading.Tests.Perf_CancellationToken

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CreateRegisterDispose - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_CancellationToken.CreateRegisterDispose.html>) 1.18 μs 1.26 μs 1.06 0.06 False
[Cancel - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_CancellationToken.Cancel.html>) 1.52 μs 1.67 μs 1.09 0.05 False
[CancelAfter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Threading.Tests.Perf_CancellationToken.CancelAfter.html>) 1.93 μs 2.10 μs 1.09 0.04 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_CancellationToken*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Threading.Tests.Perf_CancellationToken.CreateRegisterDispose ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.261077522424496 > 1.2592159154176512. IsChangePoint: Marked as a change because one of 10/11/2022 1:45:10 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.567303474997438 (T) = (0 -1291.3220528474512) / Math.Sqrt((1162.5157235502923 / (44)) + (559.9626484160198 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07703213894266561 = (1198.9633420922391 - 1291.3220528474512) / 1198.9633420922391 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tests.Perf_CancellationToken.Cancel ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6658067623849189 > 1.6008378072441136. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 9:29:01 AM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.917493512496947 (T) = (0 -1692.0904353742492) / Math.Sqrt((421.980599140767 / (43)) + (676.1984744680052 / (16))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (16) - 2, .025) and -0.10311098789876502 = (1533.9258278964185 - 1692.0904353742492) / 1533.9258278964185 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Threading.Tests.Perf_CancellationToken.CancelAfter ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.0979339021576875 > 2.055729214788499. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.449345413252109 (T) = (0 -2116.0121519990903) / Math.Sqrt((2929.250008318271 / (44)) + (876.7879104780606 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06439721372408815 = (1987.9910664136712 - 2116.0121519990903) / 1987.9910664136712 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ActivityAllocations - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat%3a%20Hierarchical).html>) 3.77 μs 4.33 μs 1.15 0.09 False
[ActivityAllocations - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat%3a%20W3C).html>) 4.78 μs 5.55 μs 1.16 0.05 False
[EnumerateActivityEventsLarge - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge.html>) 123.26 μs 130.65 μs 1.06 0.05 False
[EnumerateActivityTagsSmall - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall.html>) 733.90 ns 832.29 ns 1.13 0.03 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Activity*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 4.333193751163224 > 3.9463216922559803. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.516595935618824 (T) = (0 -4375.1269461561305) / Math.Sqrt((27659.711514739352 / (44)) + (8854.304347110501 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18446937343811634 = (3693.7442573602457 - 4375.1269461561305) / 3693.7442573602457 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.554175512186507 > 5.061026993310664. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.14034221944357 (T) = (0 -5564.166865577823) / Math.Sqrt((30178.06313717086 / (44)) + (7227.084595610786 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.17405754514519775 = (4739.262473620654 - 5564.166865577823) / 4739.262473620654 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.EnumerateActivityEventsLarge ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.6478753689492 > 129.82413246186024. IsChangePoint: Marked as a change because one of 11/15/2022 8:48:03 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.257823306425136 (T) = (0 -131114.93222892212) / Math.Sqrt((4577534.1944020605 / (44)) + (4023356.955217227 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.05012646403160211 = (124856.32609005118 - 131114.93222892212) / 124856.32609005118 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Diagnostics.Perf_Activity.EnumerateActivityTagsSmall ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 832.2856175799085 > 771.5685912991316. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -37.90289313972863 (T) = (0 -833.4534467517657) / Math.Sqrt((202.89695468588374 / (44)) + (29.49497404932715 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1320337568160074 = (736.2443405362418 - 833.4534467517657) / 736.2443405362418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.TryAddDefaultSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.TryAddDefaultSize(Int32).ConcurrentDictionary(Count%3a%20512).html>) 408.64 μs 450.31 μs 1.10 0.02 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryAddDefaultSize&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryAddDefaultSize<Int32>.ConcurrentDictionary(Count: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 450.30841849489804 > 429.1990196283285. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -20.329017133826476 (T) = (0 -450145.0335153598) / Math.Sqrt((28686578.75689623 / (44)) + (40939617.3105424 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09056394551425087 = (412763.5388707957 - 450145.0335153598) / 412763.5388707957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Linq.Tests.Perf_OrderBy

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[OrderByString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_OrderBy.OrderByString(NumberOfPeople%3a%20512).html>) 2.58 ms 3.06 ms 1.19 0.09 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_OrderBy*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_OrderBy.OrderByString(NumberOfPeople: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0641995176282055 > 2.723446670729167. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -67.28791207700004 (T) = (0 -3047326.025441087) / Math.Sqrt((852836992.6667513 / (44)) + (449125136.0127825 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18354048146036792 = (2574754.3689261884 - 3047326.025441087) / 2574754.3689261884 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Tests.Perf_Encoding.GetString(size%3a%2016%2c%20encName%3a%20%22utf-8%22).html>) 496.99 ns 592.00 ns 1.19 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 592.0045705960254 > 574.1916174245581. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 11:38:23 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.548689216699666 (T) = (0 -604.8640014730248) / Math.Sqrt((2421.2622018443203 / (44)) + (169.77742623554363 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.18416064063390108 = (510.79556330231594 - 604.8640014730248) / 510.79556330231594 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(String%2c%20String).SortedDictionary(Size%3a%20512).html>) 3.14 ms 3.52 ms 1.12 0.05 False
[IDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(String%2c%20String).IDictionary(Size%3a%20512).html>) 202.55 μs 216.26 μs 1.07 0.05 False
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(String%2c%20String).ImmutableSortedDictionary(Size%3a%20512).html>) 2.87 ms 3.31 ms 1.15 0.09 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.ContainsKeyFalse(String%2c%20String).SortedList(Size%3a%20512).html>) 2.67 ms 3.11 ms 1.16 0.08 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<String, String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.5163508325 > 3.2814812205000004. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.964051786073078 (T) = (0 -3539965.7012066543) / Math.Sqrt((2355913918.0682287 / (44)) + (4015314315.0423536 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14468016046094748 = (3092536.9579054792 - 3539965.7012066543) / 3092536.9579054792 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyFalse<String, String>.IDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 216.26378350456625 > 212.82520571838018. IsChangePoint: Marked as a change because one of 9/24/2022 2:54:54 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -12.257882627250734 (T) = (0 -215665.98474276517) / Math.Sqrt((7063752.2609918155 / (44)) + (13387483.789744928 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.061937691771084484 = (203087.23045989688 - 215665.98474276517) / 203087.23045989688 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyFalse<String, String>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.3101416641666668 > 3.02402418828125. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.51569520368917 (T) = (0 -3308649.341096306) / Math.Sqrt((2115043330.4576924 / (44)) + (2934361839.998418 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14919144379679553 = (2879110.6642466043 - 3308649.341096306) / 2879110.6642466043 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.ContainsKeyFalse<String, String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.1103027798611116 > 2.8235045819010414. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -25.69432799951375 (T) = (0 -3098657.0924732406) / Math.Sqrt((1491618478.2250059 / (44)) + (3368802490.971129 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15382350078379847 = (2685555.538059596 - 3098657.0924732406) / 2685555.538059596 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Int32).ToArray(Size%3a%20512).html>) 1.21 μs 1.77 μs 1.46 0.02 False
[SequenceCompareToDifferent - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Int32).SequenceCompareToDifferent(Size%3a%20512).html>) 159.05 ns 220.16 ns 1.38 0.12 False
[GetPinnableReference - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Span(Int32).GetPinnableReference(Size%3a%20512).html>) 44.00 ns 47.05 ns 1.07 0.07 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7678224757233671 > 1.2704817397526367. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.758984480903221 (T) = (0 -1786.5835127409912) / Math.Sqrt((102864.46273976791 / (44)) + (365.56628734625247 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.41380506911502196 = (1263.6703261074822 - 1786.5835127409912) / 1263.6703261074822 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 220.15901020878206 > 168.6501368870907. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -53.25769957157382 (T) = (0 -220.2470867398794) / Math.Sqrt((15.249105048952888 / (44)) + (12.980333214775856 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.36277171046200646 = (161.6170082259862 - 220.2470867398794) / 161.6170082259862 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Span<Int32>.GetPinnableReference(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.04535740656741 > 46.29150321807158. IsChangePoint: Marked as a change because one of 11/15/2022 10:02:59 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -11.323039769934892 (T) = (0 -47.338556714883346) / Math.Sqrt((1.0494059518166146 / (44)) + (0.3932765099460597 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.0565483567930764 = (44.80491253478381 - 47.338556714883346) / 44.80491253478381 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(Int32).Span(Size%3a%20512).html>) 4.78 μs 6.50 μs 1.36 0.07 False
[ReadOnlySpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(Int32).ReadOnlySpan(Size%3a%20512).html>) 4.78 μs 6.51 μs 1.36 0.09 False
[ImmutableArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IterateFor(Int32).ImmutableArray(Size%3a%20512).html>) 5.64 μs 7.36 μs 1.31 0.08 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IterateFor<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.502461780370793 > 5.017538202723608. IsChangePoint: Marked as a change because one of 10/11/2022 10:51:47 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -135.815735027325 (T) = (0 -6505.13608148708) / Math.Sqrt((6893.526387110876 / (44)) + (9.844457742524188 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.35478604882163584 = (4801.596596853879 - 6505.13608148708) / 4801.596596853879 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<Int32>.ReadOnlySpan(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.507467469563392 > 5.0211857852445. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.321449260703204 (T) = (0 -6505.834156043342) / Math.Sqrt((64194.05994603011 / (44)) + (37.70100834338622 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.34149452254124607 = (4849.691181533178 - 6505.834156043342) / 4849.691181533178 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IterateFor<Int32>.ImmutableArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.3647885252466425 > 5.914611771882568. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.92981099610624 (T) = (0 -7549.981825982308) / Math.Sqrt((4558.061503199645 / (44)) + (187927.25961132074 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3369430276222176 = (5647.19787604571 - 7549.981825982308) / 5647.19787604571 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).Array(Size%3a%20512).html>) 8.64 μs 12.92 μs 1.49 0.11 False
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).Span(Size%3a%20512).html>) 7.57 μs 9.21 μs 1.22 0.16 False
[ConcurrentDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).ConcurrentDictionary(Size%3a%20512).html>) 410.26 μs 450.37 μs 1.10 0.07 False
[SortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).SortedDictionary(Size%3a%20512).html>) 2.60 ms 2.90 ms 1.12 0.08 False
[IList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).IList(Size%3a%20512).html>) 52.23 μs 56.46 μs 1.08 0.07 False
[SortedList - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).SortedList(Size%3a%20512).html>) 2.16 ms 2.47 ms 1.14 0.08 False
[Dictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).Dictionary(Size%3a%20512).html>) 232.85 μs 252.40 μs 1.08 0.03 False
[List - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSet(String).List(Size%3a%20512).html>) 23.56 μs 28.85 μs 1.22 0.08 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSet<String>.Array(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.919003359399936 > 9.099196459518502. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.44277465852807 (T) = (0 -13131.475326302014) / Math.Sqrt((84792.4482972532 / (44)) + (181212.02573817753 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4891179016140548 = (8818.291226012936 - 13131.475326302014) / 8818.291226012936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.207852551549667 > 7.963030552063697. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.9539111980468 (T) = (0 -9365.408937468044) / Math.Sqrt((95185.07081227533 / (44)) + (151779.37868811408 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2149994242700053 = (7708.1591566143825 - 9365.408937468044) / 7708.1591566143825 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.ConcurrentDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 450.3660714285715 > 430.5411890662955. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.721598509578605 (T) = (0 -442097.9715005917) / Math.Sqrt((53897905.116776444 / (44)) + (20700067.342195783 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09045404584095107 = (405425.58687986585 - 442097.9715005917) / 405425.58687986585 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.SortedDictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9013805548611122 > 2.698332305859375. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -21.317294636753825 (T) = (0 -2897459.6175814006) / Math.Sqrt((3527164560.8121643 / (44)) + (2795497428.406298 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13650856798803968 = (2549439.3084169803 - 2897459.6175814006) / 2549439.3084169803 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.IList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.4639471860495 > 55.00289663815282. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.393320018480265 (T) = (0 -56317.73105032875) / Math.Sqrt((1617100.337926968 / (44)) + (1667951.2459021288 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06855293812519406 = (52704.671000334136 - 56317.73105032875) / 52704.671000334136 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.SortedList(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.465967857142857 > 2.254413983828125. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.763728376567812 (T) = (0 -2479211.1091572545) / Math.Sqrt((1146594856.3750005 / (44)) + (2457877647.2258906 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15962570569859266 = (2137940.8001857847 - 2479211.1091572545) / 2137940.8001857847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.Dictionary(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 252.39532930107526 > 245.3456672831157. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.910916204093365 (T) = (0 -252230.0060992283) / Math.Sqrt((8015290.020090203 / (44)) + (13637557.951865261 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07531572611451855 = (234563.67276485494 - 252230.0060992283) / 234563.67276485494 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Collections.IndexerSet<String>.List(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 28.84543281731361 > 24.976301221271942. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -58.27391859229857 (T) = (0 -28964.403606003405) / Math.Sqrt((133227.22606786407 / (44)) + (69875.43970917315 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21416273354788234 = (23855.454302543993 - 28964.403606003405) / 23855.454302543993 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Numerics.Tests.Constructor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SpanCastBenchmark_Int64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Int64.html>) 65.56 ns 104.50 ns 1.59 0.10 False
[SpanCastBenchmark_Byte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Byte.html>) 58.18 ns 94.99 ns 1.63 0.12 False
[ConstructorBenchmark_UInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt64.html>) 70.93 ns 127.35 ns 1.80 0.08 False
[ConstructorBenchmark_Int32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Int32.html>) 72.06 ns 129.15 ns 1.79 0.14 False
[ConstructorBenchmark_Double - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Double.html>) 71.88 ns 130.67 ns 1.82 0.11 False
[ConstructorBenchmark_Int16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Int16.html>) 74.76 ns 131.02 ns 1.75 0.11 False
[ConstructorBenchmark_Single - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Single.html>) 71.05 ns 130.06 ns 1.83 0.11 False
[ConstructorBenchmark_UInt16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt16.html>) 70.70 ns 133.39 ns 1.89 0.11 False
[SpanCastBenchmark_Int32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Int32.html>) 66.09 ns 105.36 ns 1.59 0.11 False
[ConstructorBenchmark_UInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt32.html>) 70.85 ns 131.58 ns 1.86 0.11 False
[SpanCastBenchmark_Int16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Int16.html>) 67.98 ns 104.90 ns 1.54 0.08 False
[SpanCastBenchmark_Double - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Double.html>) 66.81 ns 111.34 ns 1.67 0.11 False
[SpanCastBenchmark_UInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt32.html>) 64.86 ns 106.63 ns 1.64 0.10 False
[SpanCastBenchmark_Single - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_Single.html>) 65.60 ns 104.06 ns 1.59 0.06 False
[ConstructorBenchmark_Byte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Byte.html>) 70.93 ns 129.38 ns 1.82 0.09 False
[SpanCastBenchmark_UInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt64.html>) 66.16 ns 105.40 ns 1.59 0.13 False
[SpanCastBenchmark_SByte - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_SByte.html>) 58.97 ns 94.85 ns 1.61 0.11 False
[SpanCastBenchmark_UInt16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt16.html>) 66.78 ns 101.89 ns 1.53 0.09 False
[ConstructorBenchmark_Int64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Numerics.Tests.Constructor.ConstructorBenchmark_Int64.html>) 71.19 ns 127.57 ns 1.79 0.11 False

graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Constructor*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Constructor.SpanCastBenchmark_Int64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.50198072301959 > 70.81130545594422. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.59908337170954 (T) = (0 -102.26964368628326) / Math.Sqrt((3.427009519661027 / (44)) + (1.6327483963822793 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.526225369332162 = (67.00821893101796 - 102.26964368628326) / 67.00821893101796 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Byte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 94.9902646324341 > 62.36746136689559. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -81.86943301223799 (T) = (0 -93.85621762751332) / Math.Sqrt((1.9230900267180242 / (44)) + (2.0719939347313603 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5922696757781716 = (58.94492563368328 - 93.85621762751332) / 58.94492563368328 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.3528758220304 > 76.41746321689621. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.697644289966117 (T) = (0 -130.74765707879277) / Math.Sqrt((124.6826723410023 / (44)) + (8.970791253196003 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7264331770357297 = (75.73282234026881 - 130.74765707879277) / 75.73282234026881 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Int32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.15197696733884 > 77.60108588398477. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.228982324580706 (T) = (0 -129.99471370473168) / Math.Sqrt((119.82174059503592 / (44)) + (1.564596292379393 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7149503782160553 = (75.80086010416011 - 129.99471370473168) / 75.80086010416011 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Double ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.67373871719522 > 76.13439457377659. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.841631802626313 (T) = (0 -130.43928163720653) / Math.Sqrt((142.0891580766288 / (44)) + (4.940333243884199 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7156001534185852 = (76.03128349999685 - 130.43928163720653) / 76.03128349999685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Int16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 131.0197779567594 > 77.34209626003819. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -31.740189446285907 (T) = (0 -130.30376011565758) / Math.Sqrt((122.66313542930806 / (44)) + (2.227351293861295 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7164420922835526 = (75.91503418696847 - 130.30376011565758) / 75.91503418696847 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Single ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 130.06467975189477 > 74.87073761835128. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.50272594594233 (T) = (0 -129.90252833617504) / Math.Sqrt((145.88271506861878 / (44)) + (2.1878952020137583 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7317248810338949 = (75.0133752531286 - 129.90252833617504) / 75.0133752531286 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 133.38938158335364 > 75.06807647928227. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -32.380864761682524 (T) = (0 -129.75174348699505) / Math.Sqrt((109.90033548534835 / (44)) + (4.577722913440286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7176565264906135 = (75.53998222921439 - 129.75174348699505) / 75.53998222921439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Int32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.36000323124675 > 69.81869397046569. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -73.93713010742547 (T) = (0 -103.36121809207373) / Math.Sqrt((2.883622853954203 / (44)) + (2.5564384919827394 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.553592413129661 = (66.53046012490235 - 103.36121809207373) / 66.53046012490235 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 131.58015727211702 > 75.4554282801025. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -28.26834262833134 (T) = (0 -131.65004829413667) / Math.Sqrt((125.11443745101388 / (44)) + (16.850035576120707 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7472232452204646 = (75.34815522530727 - 131.65004829413667) / 75.34815522530727 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Int16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.90201332671903 > 70.19042991129741. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -80.62568376401023 (T) = (0 -103.90184055371407) / Math.Sqrt((2.883489963254961 / (44)) + (2.202369507305795 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.556637537227426 = (66.74761340959101 - 103.90184055371407) / 66.74761340959101 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Double ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 111.33867246242677 > 70.32521376649157. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.99056770450468 (T) = (0 -103.20358405910883) / Math.Sqrt((2.3213720602826577 / (44)) + (5.583470659402237 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5472189676933987 = (66.70263628745789 - 103.20358405910883) / 66.70263628745789 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt32 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 106.62806212456992 > 69.27741467277701. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.87524104103628 (T) = (0 -105.56986923333632) / Math.Sqrt((1.0351739560659676 / (44)) + (81.36867883023433 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.595167131365999 = (66.18107103481567 - 105.56986923333632) / 66.18107103481567 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Single ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 104.05863745849102 > 68.91588506475091. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -83.38921699141447 (T) = (0 -103.8424823369991) / Math.Sqrt((1.1858929523419 / (44)) + (2.65488743566047 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5690033423389008 = (66.1837228352885 - 103.8424823369991) / 66.1837228352885 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Byte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 129.38444454636107 > 74.5031741098795. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.44974973822346 (T) = (0 -130.39457100996088) / Math.Sqrt((124.9034571901857 / (44)) + (5.757999066909469 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.7270689943050965 = (75.50049907671838 - 130.39457100996088) / 75.50049907671838 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 105.39713700433047 > 69.26471140999872. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.09772498277107 (T) = (0 -103.41875636902367) / Math.Sqrt((19.47802626449258 / (44)) + (3.981963899087346 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5400835061805148 = (67.15139533278129 - 103.41875636902367) / 67.15139533278129 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_SByte ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 94.85003248266506 > 62.30509904017331. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -64.07469515389741 (T) = (0 -93.93654419203573) / Math.Sqrt((6.582366881350537 / (44)) + (2.120596508446137 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5821300281603945 = (59.37346647876944 - 93.93654419203573) / 59.37346647876944 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt16 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 101.88533278346448 > 70.0691197706884. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -72.10523967088655 (T) = (0 -103.2534323527112) / Math.Sqrt((1.2693341683432215 / (44)) + (3.526514715232377 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5595161231878011 = (66.20863408686743 - 103.2534323527112) / 66.20863408686743 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Int64 ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 127.56946692740128 > 77.02784558276483. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.203796457481854 (T) = (0 -130.10973225724106) / Math.Sqrt((154.9336733575985 / (44)) + (9.096520217324468 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6925404949223213 = (76.87244863421269 - 130.10973225724106) / 76.87244863421269 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Guid

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryWriteBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.TryWriteBytes.html>) 52.37 ns 96.27 ns 1.84 0.09 False
[ctor_str - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.ctor_str.html>) 1.64 μs 3.08 μs 1.87 0.03 False
[NotEqualsOperator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.NotEqualsOperator.html>) 65.14 ns 80.81 ns 1.24 0.06 False
[ctor_bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.ctor_bytes.html>) 104.14 ns 163.80 ns 1.57 0.09 False
[GuidToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.GuidToString.html>) 1.13 μs 1.21 μs 1.07 0.02 False
[ParseExactD - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.ParseExactD.html>) 1.62 μs 2.90 μs 1.79 0.04 False
[EqualsSame - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.EqualsSame.html>) 47.79 ns 75.51 ns 1.58 0.10 False
[EqualsNotSame - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Guid.EqualsNotSame.html>) 30.88 ns 58.25 ns 1.89 0.17 False

graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Guid*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Guid.TryWriteBytes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 96.27158445609292 > 55.397712893029386. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -18.96333827380124 (T) = (0 -96.36234029252984) / Math.Sqrt((176.8281455318089 / (44)) + (1.122264725684083 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6615828341186389 = (57.994304174214555 - 96.36234029252984) / 57.994304174214555 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.ctor_str ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.0825522886975456 > 1.7490753078335706. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -98.30044550768929 (T) = (0 -3041.504098183182) / Math.Sqrt((767.6103550503817 / (44)) + (2422.7031550693328 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.8053053460620362 = (1684.7588164615483 - 3041.504098183182) / 1684.7588164615483 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.NotEqualsOperator ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 80.80770003354546 > 68.55334332837477. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -55.42969399616112 (T) = (0 -80.51576535597572) / Math.Sqrt((0.7512903899864011 / (44)) + (0.9128865782763896 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2460664817139608 = (64.61594669108386 - 80.51576535597572) / 64.61594669108386 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.ctor_bytes ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 163.7982806978753 > 111.38041262053837. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.246164085606917 (T) = (0 -162.56022060452216) / Math.Sqrt((157.78576344889817 / (44)) + (15.921784544832887 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.4739312241032659 = (110.29023467728162 - 162.56022060452216) / 110.29023467728162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.GuidToString ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.209375471377439 > 1.1937533354471281. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/8/2022 9:11:55 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -19.857574910272504 (T) = (0 -1223.1410517867712) / Math.Sqrt((854.0714522553948 / (44)) + (76.27424110996132 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08737203190434036 = (1124.8597682291454 - 1223.1410517867712) / 1124.8597682291454 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.ParseExactD ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.902973867102397 > 1.7196689197300257. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -161.26019679358254 (T) = (0 -2906.5812890387283) / Math.Sqrt((521.0940623083161 / (44)) + (731.4562209322777 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.7603217860114116 = (1651.1647541581274 - 2906.5812890387283) / 1651.1647541581274 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.EqualsSame ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 75.51299919072144 > 50.22852065322537. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -65.75857292252802 (T) = (0 -76.48438974401567) / Math.Sqrt((2.8668721840446305 / (44)) + (1.7368732997645708 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5766063608215889 = (48.51203930457233 - 76.48438974401567) / 48.51203930457233 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Guid.EqualsNotSame ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 58.24830951728019 > 32.30349575817946. IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -113.68285726816235 (T) = (0 -58.59856414405806) / Math.Sqrt((0.6136849530811046 / (44)) + (0.6718243974082163 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.887413602680209 = (31.047018025537998 - 58.59856414405806) / 31.047018025537998 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromReader(Mode%3a%20SourceGen).html>) 17.56 μs 19.92 μs 1.13 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 11.33 μs 12.76 μs 1.13 0.02 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromString(Mode%3a%20Reflection).html>) 14.08 μs 15.66 μs 1.11 0.01 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromReader(Mode%3a%20Reflection).html>) 16.69 μs 18.96 μs 1.14 0.01 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 10.34 μs 11.62 μs 1.12 0.04 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(SimpleStructWithProperties).DeserializeFromString(Mode%3a%20SourceGen).html>) 14.83 μs 17.18 μs 1.16 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;SimpleStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 19.924564216624148 > 18.53027779839456. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -54.29521787470199 (T) = (0 -20036.445389753517) / Math.Sqrt((27798.459310284503 / (44)) + (19589.334991284286 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13544204286160838 = (17646.38320002356 - 20036.445389753517) / 17646.38320002356 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 12.764227151639343 > 11.845627164300033. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.61339997611277 (T) = (0 -12797.010047124297) / Math.Sqrt((26638.404290169492 / (44)) + (11675.046042395046 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13385965156546906 = (11286.238141957023 - 12797.010047124297) / 11286.238141957023 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.655391165162152 > 14.744098870789442. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.28394389831145 (T) = (0 -15814.015620745211) / Math.Sqrt((22046.833046892363 / (44)) + (38360.50936468176 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13173005014179984 = (13973.31070140251 - 15814.015620745211) / 13973.31070140251 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 18.961599541064906 > 17.56407146997862. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.188493034662706 (T) = (0 -19002.586331927614) / Math.Sqrt((36335.082089891956 / (44)) + (28848.765497798817 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13528562930543256 = (16738.1545590015 - 19002.586331927614) / 16738.1545590015 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.624208278401927 > 10.94296439373918. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.65646097305665 (T) = (0 -11784.486464494154) / Math.Sqrt((19771.39939826128 / (44)) + (9348.726621553213 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13807944118756607 = (10354.71342158439 - 11784.486464494154) / 10354.71342158439 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<SimpleStructWithProperties>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 17.18089442872859 > 15.788441004297686. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.050914938267105 (T) = (0 -16885.556610162654) / Math.Sqrt((30128.97975689828 / (44)) + (16098.331038287815 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12595492960302904 = (14996.654098859792 - 16885.556610162654) / 14996.654098859792 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDeepUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 237.37 ms 266.08 ms 1.12 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 266.0780004 > 251.39922507. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/25/2022 8:17:52 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.133407999086643 (T) = (0 -257972650.18476188) / Math.Sqrt((24923402736112.246 / (44)) + (13341461904059.39 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.06544819451491714 = (242125944.2860223 - 257972650.18476188) / 242125944.2860223 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ToString(value%3a%209223372036854775807).html>) 764.68 ns 967.84 ns 1.27 0.06 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.Parse(value%3a%20%22-9223372036854775808%22).html>) 803.36 ns 880.97 ns 1.10 0.04 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ParseSpan(value%3a%20%22-9223372036854775808%22).html>) 864.61 ns 952.28 ns 1.10 0.05 False
[Parse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.Parse(value%3a%20%229223372036854775807%22).html>) 791.34 ns 869.78 ns 1.10 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryFormat(value%3a%20-9223372036854775808).html>) 905.17 ns 1.08 μs 1.20 0.07 False
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParse(value%3a%20%229223372036854775807%22).html>) 753.89 ns 831.96 ns 1.10 0.06 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ToString(value%3a%20-9223372036854775808).html>) 906.47 ns 1.08 μs 1.19 0.04 False
[ParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.ParseSpan(value%3a%20%229223372036854775807%22).html>) 847.02 ns 923.70 ns 1.09 0.05 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryFormat(value%3a%209223372036854775807).html>) 735.67 ns 959.05 ns 1.30 0.08 False
[TryParseSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryParseSpan(value%3a%20%22-9223372036854775808%22).html>) 757.97 ns 839.18 ns 1.11 0.07 False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Int64.TryFormat(value%3a%2012345).html>) 348.09 ns 389.71 ns 1.12 0.15 False

graph graph graph graph graph graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.ToString(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 967.8382526564345 > 811.5471161933109. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -29.346931965333116 (T) = (0 -977.0975287007166) / Math.Sqrt((885.404297950699 / (44)) + (498.5708863190296 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28106557745910643 = (762.7224912550638 - 977.0975287007166) / 762.7224912550638 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.Parse(value: "-9223372036854775808") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 880.9676312341754 > 839.8350056358971. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/23/2022 1:14:54 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.352145047307683 (T) = (0 -890.4463569076531) / Math.Sqrt((159.35897287033214 / (44)) + (123.36732479692965 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10390916477796365 = (806.6300972206842 - 890.4463569076531) / 806.6300972206842 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 952.2757752693132 > 892.5342862017782. IsChangePoint: Marked as a change because one of 10/18/2022 1:35:00 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -22.41334822277467 (T) = (0 -944.3857770585101) / Math.Sqrt((113.8386053189746 / (44)) + (204.36469782334058 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10565478675617791 = (854.1416257321995 - 944.3857770585101) / 854.1416257321995 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.Parse(value: "9223372036854775807") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 869.7838676080611 > 835.0703460257848. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.013683521004932 (T) = (0 -878.2712098727167) / Math.Sqrt((1136.5686155057147 / (44)) + (80.95087958185405 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09088348003838383 = (805.1008434391305 - 878.2712098727167) / 805.1008434391305 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0826484497990536 > 952.0136900553856. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.376043629184526 (T) = (0 -1107.0324723269748) / Math.Sqrt((884.7242142372156 / (44)) + (2494.7314035839872 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.21552184620894768 = (910.7466688317146 - 1107.0324723269748) / 910.7466688317146 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.TryParse(value: "9223372036854775807") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 831.9603625072212 > 792.5807137027139. IsChangePoint: Marked as a change because one of 10/11/2022 10:45:23 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.54102790660888 (T) = (0 -831.3411375621147) / Math.Sqrt((77.8518069160536 / (44)) + (76.34969311541303 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10645857920714213 = (751.3531488524685 - 831.3411375621147) / 751.3531488524685 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.ToString(value: -9223372036854775808) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0799793798039503 > 951.7484194957716. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.659435974483564 (T) = (0 -1089.5328939876608) / Math.Sqrt((763.6521257986554 / (44)) + (140.4902903730252 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.19974287293416707 = (908.1386675155071 - 1089.5328939876608) / 908.1386675155071 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 923.7002898506117 > 891.472449212378. IsChangePoint: Marked as a change because one of 10/18/2022 9:26:37 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.16817691092804 (T) = (0 -930.3927734416393) / Math.Sqrt((197.81668950311067 / (44)) + (65.28064621336117 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.09512085715684956 = (849.5799959989101 - 930.3927734416393) / 849.5799959989101 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 959.0487180511183 > 786.5827507144976. IsChangePoint: Marked as a change because one of 10/11/2022 2:16:14 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -45.752363666116125 (T) = (0 -974.2968570695191) / Math.Sqrt((279.9806029574343 / (44)) + (296.0462611015816 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.31562498979191206 = (740.5581868915551 - 974.2968570695191) / 740.5581868915551 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 839.1806811622539 > 801.6932574605312. IsChangePoint: Marked as a change because one of 10/22/2022 11:45:45 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.315195675112605 (T) = (0 -849.1738838350655) / Math.Sqrt((121.50365125589292 / (44)) + (363.9282294911627 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11095916356969125 = (764.3610239520709 - 849.1738838350655) / 764.3610239520709 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Int64.TryFormat(value: 12345) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 389.7050631981953 > 362.15483050484585. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.580008611839812 (T) = (0 -390.9426824033135) / Math.Sqrt((53.30783420102266 / (44)) + (69.6790332355729 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12211655798418779 = (348.39757030732846 - 390.9426824033135) / 348.39757030732846 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[CallBlockCopy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements%3a%201000).html>) 696.14 ns 1.26 μs 1.81 0.11 False
[CallBlockCopy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements%3a%20100).html>) 314.49 ns 386.65 ns 1.23 0.08 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 1000) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2601367387754543 > 732.7635271444873. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -9.313130486593229 (T) = (0 -1259.7188414658751) / Math.Sqrt((95009.67012506972 / (44)) + (793.1993249996768 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.5330908524591054 = (821.6857073051237 - 1259.7188414658751) / 821.6857073051237 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 386.6539452800313 > 332.6992951207662. IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -4.139418514179559 (T) = (0 -387.5225035491755) / Math.Sqrt((5047.010901088529 / (44)) + (13.958168523737148 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1298132901120953 = (342.99694200864565 - 387.5225035491755) / 342.99694200864565 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(LoginViewModel).SystemTextJsonReflection.html>) 21.36 μs 24.29 μs 1.14 0.01 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(LoginViewModel).SystemTextJsonSourceGen.html>) 21.58 μs 23.86 μs 1.11 0.01 False
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(LoginViewModel).JsonNet.html>) 37.79 μs 40.44 μs 1.07 0.00 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.290025687027143 > 23.03320584325728. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.140948816089455 (T) = (0 -24204.762112325217) / Math.Sqrt((94636.94600379617 / (44)) + (27438.257201778262 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11038467115882385 = (21798.537696908723 - 24204.762112325217) / 21798.537696908723 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.85843384698826 > 22.728003283545867. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.23920331467686 (T) = (0 -24029.8913629293) / Math.Sqrt((76416.94168302728 / (44)) + (16858.46502272972 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.10922086318663675 = (21663.75711136083 - 24029.8913629293) / 21663.75711136083 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<LoginViewModel>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 40.43783359173126 > 39.708328249531085. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -27.560982493413317 (T) = (0 -40599.80777339836) / Math.Sqrt((177888.26120957633 / (44)) + (85695.093743553 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.07346687142343555 = (37821.20236236291 - 40599.80777339836) / 37821.20236236291 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromReader(Mode%3a%20SourceGen).html>) 49.70 μs 57.44 μs 1.16 0.04 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20Reflection).html>) 29.92 μs 35.01 μs 1.17 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromString(Mode%3a%20Reflection).html>) 37.18 μs 41.83 μs 1.13 0.01 False
[DeserializeFromString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromString(Mode%3a%20SourceGen).html>) 39.32 μs 45.14 μs 1.15 0.06 False
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromReader(Mode%3a%20Reflection).html>) 47.40 μs 55.16 μs 1.16 0.02 False
[DeserializeFromUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.ReadJson(LargeStructWithProperties).DeserializeFromUtf8Bytes(Mode%3a%20SourceGen).html>) 32.53 μs 37.56 μs 1.15 0.01 False

graph graph graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.43965092218457 > 52.46054870434376. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.0149817629944 (T) = (0 -57109.11510174652) / Math.Sqrt((808045.5074685642 / (43)) + (180381.248238636 / (14))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (14) - 2, .025) and -0.1506655349436691 = (49631.37711823645 - 57109.11510174652) / 49631.37711823645 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.010566134155454 > 31.67374087362537. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -33.37856217146205 (T) = (0 -34938.370373305115) / Math.Sqrt((277180.9419088537 / (44)) + (200958.65720917945 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1548406551583808 = (30253.845166642048 - 34938.370373305115) / 30253.845166642048 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 41.8334892953149 > 38.90599650572436. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -38.413240703948354 (T) = (0 -42311.49450190326) / Math.Sqrt((319544.6762906991 / (44)) + (165159.14665957852 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1398913399860912 = (37118.88407049354 - 42311.49450190326) / 37118.88407049354 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromString(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 45.13565100864553 > 41.626933000879326. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.164165930933084 (T) = (0 -45108.85363793394) / Math.Sqrt((534275.2861493308 / (44)) + (315659.25103249855 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1387158479989809 = (39613.792780000294 - 45108.85363793394) / 39613.792780000294 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromReader(Mode: Reflection) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 55.16363048245614 > 49.642996810326544. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -42.16579653221926 (T) = (0 -54781.5405606336) / Math.Sqrt((549444.7118810299 / (44)) + (279294.8124699243 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15707910591945767 = (47344.68048068517 - 54781.5405606336) / 47344.68048068517 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 37.56294176706827 > 34.11579458321358. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.31571763962391 (T) = (0 -37656.77841962299) / Math.Sqrt((266152.536355684 / (44)) + (144295.49151877122 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15475161671953785 = (32610.28421557858 - 37656.77841962299) / 32610.28421557858 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_FromStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[JsonNet_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(Location).JsonNet.html>) 73.63 μs 78.66 μs 1.07 0.02 False
[DataContractJsonSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.JsonFromStream(Location).DataContractJsonSerializer.html>) 354.67 μs 377.11 μs 1.06 0.00 False
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(Location).SystemTextJsonSourceGen.html>) 51.40 μs 57.70 μs 1.12 0.01 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_FromStream(Location).SystemTextJsonReflection.html>) 50.10 μs 56.09 μs 1.12 0.01 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<Location>.JsonNet_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 78.65539224971008 > 77.52816650967472. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -24.775191104719458 (T) = (0 -79619.68952324899) / Math.Sqrt((1197341.6529939242 / (44)) + (560970.0872326223 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08588800222098456 = (73322.19286003854 - 79619.68952324899) / 73322.19286003854 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<Location>.DataContractJsonSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 377.1055557539683 > 373.74473318481694. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -26.762761554413995 (T) = (0 -379243.5587403843) / Math.Sqrt((21910777.351339404 / (44)) + (5751656.240562556 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07095349269592702 = (354117.6730146413 - 379243.5587403843) / 354117.6730146413 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 57.70305849063299 > 53.37768466760439. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -40.07442818356562 (T) = (0 -57661.337233455466) / Math.Sqrt((352045.97784729925 / (44)) + (308300.20023973583 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13306849471427992 = (50889.54242611399 - 57661.337233455466) / 50889.54242611399 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### MicroBenchmarks.Serializers.Json_FromStream<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 56.09055761390887 > 53.00470276390024. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.099212079590785 (T) = (0 -56948.82633306473) / Math.Sqrt((609688.4940711295 / (44)) + (287337.23702904413 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13016464229551217 = (50389.8495863348 - 56948.82633306473) / 50389.8495863348 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in SIMD.RayTracerBench

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Bench - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/SIMD.RayTracerBench.Bench.html>) 5.21 secs 5.87 secs 1.13 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SIMD.RayTracerBench*'
### Payloads [Baseline]() [Compare]() ### Histogram #### SIMD.RayTracerBench.Bench ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.874281466666667 > 5.370514724925001. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -36.849875613578725 (T) = (0 -5823843394.639733) / Math.Sqrt((6037182216571426 / (44)) + (4122447584174859 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.14736597319689854 = (5075837640.899177 - 5823843394.639733) / 5075837640.899177 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(BinaryData).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 27.02 μs 31.36 μs 1.16 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 31.364859053308823 > 28.444853212732117. IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -6.97415427542305 (T) = (0 -29326.84225416773) / Math.Sqrt((586450.3893427762 / (44)) + (1219026.507748168 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.07891348237624599 = (27181.82943601466 - 29326.84225416773) / 27181.82943601466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring%3a%20ja).html>) 3.98 μs 5.26 μs 1.32 0.07 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring%3a%20fr).html>) 3.92 μs 5.24 μs 1.34 0.04 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring%3a%20).html>) 3.92 μs 5.22 μs 1.33 0.07 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring%3a%20da).html>) 4.08 μs 5.22 μs 1.28 0.05 False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.259341285706254 > 4.289794178952822. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.670110899121376 (T) = (0 -5205.288077942538) / Math.Sqrt((12139.613613614634 / (44)) + (5204.5985783270635 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2962780728689189 = (4015.5643969370008 - 5205.288077942538) / 4015.5643969370008 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.241161247026843 > 4.315003468115543. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -43.76472382370205 (T) = (0 -5206.534257597612) / Math.Sqrt((14016.234861228872 / (44)) + (5920.103010037519 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28946674873892647 = (4037.7421617808304 - 5206.534257597612) / 4037.7421617808304 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.2184811627511065 > 4.158444273704924. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -34.31715711146078 (T) = (0 -5214.4579121924035) / Math.Sqrt((20682.146353060376 / (44)) + (10434.430274451302 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.28981319315576287 = (4042.8008798966325 - 5214.4579121924035) / 4042.8008798966325 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 5.222357238556537 > 4.308636271680654. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -47.95559958805663 (T) = (0 -5165.3724099695855) / Math.Sqrt((12061.494933836271 / (44)) + (4245.240358826799 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2806380982308587 = (4033.4364697608985 - 5165.3724099695855) / 4033.4364697608985 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Constructors<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadOnlySpanImplicitCastFromArraySegment - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(String).ReadOnlySpanImplicitCastFromArraySegment.html>) 67.49 ns 76.30 ns 1.13 0.08 False
[ReadOnlySpanImplicitCastFromSpan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Constructors(String).ReadOnlySpanImplicitCastFromSpan.html>) 21.26 ns 35.45 ns 1.67 0.20 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Constructors&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromArraySegment ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 76.29529548563012 > 71.34737041113384. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 10/19/2022 7:27:15 PM, 11/12/2022 3:22:24 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -16.526579298258333 (T) = (0 -76.8009659550273) / Math.Sqrt((0.6163115262223671 / (44)) + (3.970798852716292 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.1281679940340704 = (68.07582413360676 - 76.8009659550273) / 68.07582413360676 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Memory.Constructors<String>.ReadOnlySpanImplicitCastFromSpan ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.447992995552084 > 22.04260831791313. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -44.282001092919664 (T) = (0 -35.39011079795573) / Math.Sqrt((0.9974275740162426 / (44)) + (1.2209351431825453 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.6768457324088735 = (21.10516794357466 - 35.39011079795573) / 21.10516794357466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Memory.Memory<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToArray - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Memory.Memory(Byte).ToArray(Size%3a%20512).html>) 437.92 ns 593.68 ns 1.36 0.06 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory&lt;Byte&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Memory<Byte>.ToArray(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 593.6815438312818 > 462.7865611490569. IsChangePoint: Marked as a change because one of 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -10.510018061534899 (T) = (0 -600.3509818375896) / Math.Sqrt((8091.299791614027 / (44)) + (122.00276437144818 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.3202948636026299 = (454.70977611731263 - 600.3509818375896) / 454.70977611731263 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[HashSet - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.CreateAddAndRemove(Int32).HashSet(Size%3a%20512).html>) 232.25 μs 274.22 μs 1.18 0.03 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.CreateAddAndRemove<Int32>.HashSet(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 274.2170456168831 > 246.8516319918119. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -39.32816314894456 (T) = (0 -273820.39083270886) / Math.Sqrt((18946925.18070819 / (44)) + (6812128.16316761 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.15620042995577346 = (236827.78845116234 - 273820.39083270886) / 236827.78845116234 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Xml.Linq.Perf_XElementList

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Enumerator - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Xml.Linq.Perf_XElementList.Enumerator.html>) 2.67 μs 2.96 μs 1.11 0.12 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XElementList*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Xml.Linq.Perf_XElementList.Enumerator ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.9639175598991176 > 2.799475307169099. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -13.12457463301963 (T) = (0 -2979.985838650132) / Math.Sqrt((5885.472294576356 / (44)) + (5614.989492824275 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.11021758789205865 = (2684.145766694395 - 2979.985838650132) / 2684.145766694395 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[XmlSerializer_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.XmlToStream(LoginViewModel).XmlSerializer.html>) 58.68 μs 63.24 μs 1.08 0.01 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_ ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 63.240330664401384 > 61.77235563384435. IsChangePoint: Marked as a change because one of 10/10/2022 9:53:50 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -17.760535831167815 (T) = (0 -63758.99368264897) / Math.Sqrt((262720.69441453647 / (44)) + (1108636.1716363446 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.08545670685734741 = (58739.32445195927 - 63758.99368264897) / 58739.32445195927 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Collections.IndexerSetReverse(Int32).Span(Size%3a%20512).html>) 5.52 μs 6.73 μs 1.22 0.15 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.IndexerSetReverse<Int32>.Span(Size: 512) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.733747500416252 > 5.780906465549699. IsChangePoint: Marked as a change because one of 10/22/2022 11:54:51 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -49.9335609975542 (T) = (0 -6766.297508208523) / Math.Sqrt((6477.533717210311 / (44)) + (6990.974047089773 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.2236246275945715 = (5529.716675864772 - 6766.297508208523) / 5529.716675864772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteAsciiString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars%3a%2032).html>) 919.59 ns 1.03 μs 1.12 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiString(StringLengthInChars: 32) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0282262958044301 > 960.1107046950955. IsChangePoint: Marked as a change because one of 10/11/2022 6:23:40 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -30.41700923765956 (T) = (0 -1033.172729556685) / Math.Sqrt((185.96510002507625 / (44)) + (135.51729228344524 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.12008294659776336 = (922.4073383984044 - 1033.172729556685) / 922.4073383984044 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in Benchstone.MDBenchI.MDNDhrystone

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/Benchstone.MDBenchI.MDNDhrystone.Test.html>) 7.27 secs 8.11 secs 1.12 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDNDhrystone*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchI.MDNDhrystone.Test ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.108880999733334 > 7.57777429528. IsChangePoint: Marked as a change because one of 9/29/2022 2:21:18 AM, 12/3/2022 2:55:34 AM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -14.633736259017953 (T) = (0 -8104316671.349048) / Math.Sqrt((1.4516248710021126E+17 / (44)) + (16820251499717154 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.13643257334617 = (7131366049.713169 - 8104316671.349048) / 7131366049.713169 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 9333e468a9bc61950f3c7b25d52338dfd956ce78
Compare 0b7fd637e93a02942eac1cec903d836194fceb86
Diff Diff

Regressions in System.IO.Tests.Perf_FileInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ctor_str - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_FileInfo.ctor_str.html>) 1.61 μs 3.03 μs 1.88 0.07 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.Perf_FileInfo.ctor_str ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 3.029550417886179 > 1.698266042429667. IsChangePoint: Marked as a change because one of 9/23/2022 6:26:22 PM, 12/9/2022 1:52:32 PM, 12/13/2022 2:17:54 AM falls between 12/4/2022 6:53:17 AM and 12/13/2022 2:17:54 AM. IsRegressionStdDev: Marked as regression because -83.05994650421023 (T) = (0 -3004.110352118251) / Math.Sqrt((582.1748494661001 / (44)) + (3964.4231649070653 / (15))) is less than -2.0024654592901125 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (15) - 2, .025) and -0.8539254848538719 = (1620.4051223531444 - 3004.110352118251) / 1620.4051223531444 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)