[Perf] Linux/x64: 4 Regressions on 9/27/2022 4:04:11 PM

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Ctor_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address%3a%20%5b16%2c%2032%2c%2048%2c%2064%2c%2080%2c%20...%5d).html>)	389.18 ns	793.04 ns	2.04	0.10	False
[GetAddressBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address%3a%201020%3a3040%3a5060%3a7080%3a9010%3a1112%3a1314%3a1516).html>)	443.87 ns	727.40 ns	1.64	0.08	False
[Ctor_Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address%3a%20%5b16%2c%2032%2c%2048%2c%2064%2c%2080%2c%20...%5d).html>)	365.29 ns	755.99 ns	2.07	0.06	False
[TryWriteBytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address%3a%201020%3a3040%3a5060%3a7080%3a9010%3a1112%3a1314%3a1516).html>)	434.88 ns	735.09 ns	1.69	0.07	False

graph graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 793.0396262103835 > 412.97496375910043. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsRegressionStdDev: Marked as regression because -148.05747465230505 (T) = (0 -783.4900374817038) / Math.Sqrt((79.9783756712201 / (29)) + (149.15722185030273 / (36))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (36) - 2, .025) and -0.9858166470640966 = (394.5429899784776 - 783.4900374817038) / 394.5429899784776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.GetAddressBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 727.4031845899871 > 466.349269077429. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsRegressionStdDev: Marked as regression because -70.33021988145737 (T) = (0 -748.3544321021982) / Math.Sqrt((25.334552521454054 / (29)) + (617.3962515784395 / (35))) is less than -1.998971517032675 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (35) - 2, .025) and -0.6788944995741768 = (445.74238124671064 - 748.3544321021982) / 445.74238124671064 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 755.9905045813417 > 384.91259737238437. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsRegressionStdDev: Marked as regression because -128.06071790809835 (T) = (0 -762.6907463732886) / Math.Sqrt((39.46128765588628 / (30)) + (259.8667370064361 / (32))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (32) - 2, .025) and -1.0651860876414894 = (369.3084855342535 - 762.6907463732886) / 369.3084855342535 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 735.0905385824451 > 455.8016923746357. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsRegressionStdDev: Marked as regression because -65.85639938115801 (T) = (0 -736.6458241790391) / Math.Sqrt((206.4693472473326 / (29)) + (450.4780517475538 / (34))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (34) - 2, .025) and -0.6763948337033375 = (439.4226284697554 - 736.6458241790391) / 439.4226284697554 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Linq.Tests.Perf_Enumerable.Min(input%3a%20IEnumerable).html>)	2.47 μs	2.16 μs	0.88	0.26	False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 2.1593931241785715 < 2.35453123163365. IsChangePoint: Marked as a change because one of 7/13/2022 7:22:00 AM, 9/22/2022 3:11:11 PM, 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 12.098051343033308 (T) = (0 -2187.096992844359) / Math.Sqrt((13836.091486949126 / (299)) + (6580.853245694546 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.07890469605020306 = (2374.452440986024 - 2187.096992844359) / 2374.452440986024 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	007563949e53bad52dfcdf95927b0ce00101df4d
Compare	1f688ed75133664aa20c838fa451e2ecf9f1f424
Diff	Diff

Improvements in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Ctor_Span - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address%3a%20%5b16%2c%2032%2c%2048%2c%2064%2c%2080%2c%20...%5d).html>)	37.93 ns	24.26 ns	0.64	0.52	False
[Ctor_Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompliationMode=wasm_RunKind=micro/System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address%3a%20%5b16%2c%2032%2c%2048%2c%2064%2c%2080%2c%20...%5d).html>)	39.22 ns	33.62 ns	0.86	0.52	False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 24.2637587990092 < 32.6342076654047. IsChangePoint: Marked as a change because one of 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 12.658821531369549 (T) = (0 -27.948175501777232) / Math.Sqrt((7.392196679247289 / (299)) + (5.986580804515473 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.17056751068837545 = (33.695539856381096 - 27.948175501777232) / 33.695539856381096 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Bytes(address: [16, 32, 48, 64, 80, ...]) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 33.61754181625243 < 37.067326680794366. IsChangePoint: Marked as a change because one of 7/28/2022 12:19:57 AM, 9/27/2022 11:22:50 AM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 11.083034367972369 (T) = (0 -34.11393548793784) / Math.Sqrt((11.614310841847756 / (299)) + (3.8175924939423656 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.11324863453648766 = (38.4706884213324 - 34.11393548793784) / 38.4706884213324 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

dotnet / perf-autofiling-issues