[Perf] Linux/x64: 49 Regressions on 4/17/2023 1:30:24 AM

performanceautofiler[bot] commented 1 year ago

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.BlockCopyPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[CallBlockCopy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements%3a%201000).html>)	12.86 ns	16.78 ns	1.31	0.31	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.BlockCopyPerf*'

### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 1000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.7832335517655 > 13.539240007935843. IsChangePoint: Marked as a change because one of 3/22/2023 10:50:22 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -10.964259881430495 (T) = (0 -16.589723775383348) / Math.Sqrt((2.825292120996728 / (31)) + (0.0216274314174295 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.2553815633801119 = (13.21488562466662 - 16.589723775383348) / 13.21488562466662 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%204).html>)	2.91 ns	7.08 ns	2.44	0.18	True
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%200).html>)	1.35 ns	6.22 ns	4.60	0.06	True
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%2012345).html>)	7.90 ns	10.14 ns	1.28	0.07	False
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%200).html>)	1.40 ns	6.65 ns	4.75	0.13	True
[FormatterDecimal - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value%3a%20123456.789).html>)	30.38 ns	47.97 ns	1.58	0.01	True

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 4) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.07959809686284 > 3.064968526731939. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -71.51004251426706 (T) = (0 -7.075231593721525) / Math.Sqrt((0.1161981049872872 / (30)) + (6.978103917374816E-05 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -1.7024459621675556 = (2.6180843919804713 - 7.075231593721525) / 2.6180843919804713 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.218835235764051 > 1.4701949400485947. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -80.12529850429289 (T) = (0 -6.225005808260373) / Math.Sqrt((0.12281148717940028 / (30)) + (5.092790659291841E-05 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -4.694794993219447 = (1.0931044604190714 - 6.225005808260373) / 1.0931044604190714 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 10.137880336077373 > 8.211119302675476. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -4.042807133860763 (T) = (0 -10.14075116355823) / Math.Sqrt((1.7348515370988993 / (30)) + (0.00034876946376687773 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.10609475767794309 = (9.168067286429423 - 10.14075116355823) / 9.168067286429423 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.646906298876834 > 1.4414436670766893. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -88.10081408586912 (T) = (0 -6.6511443969098005) / Math.Sqrt((0.1188117990086398 / (30)) + (4.651406710812904E-05 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -5.038861139385512 = (1.1013905177469592 - 6.6511443969098005) / 1.1013905177469592 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.969122146369344 > 31.798300179699385. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -222.4105759903511 (T) = (0 -47.75822939532259) / Math.Sqrt((0.06472756341416012 / (30)) + (0.023902979058749045 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.574696000997676 = (30.328539200623187 - 47.75822939532259) / 30.328539200623187 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>)	1.78 ms	2.19 ms	1.23	0.01	True
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>)	1.67 ms	1.78 ms	1.07	0.01	False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%2010).html>)	498.94 ns	529.71 ns	1.06	0.00	False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>)	1.69 ms	1.90 ms	1.12	0.01	False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>)	1.89 ms	2.25 ms	1.19	0.00	True
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>)	1.65 ms	1.76 ms	1.07	0.01	False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20DataSize%3a%20100000).html>)	1.82 ms	2.19 ms	1.20	0.02	False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>)	1.73 ms	1.87 ms	1.08	0.01	False
[WriteBasicUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%20100000).html>)	1.85 ms	2.30 ms	1.24	0.03	False
[WriteBasicUtf16 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20DataSize%3a%2010).html>)	676.84 ns	719.92 ns	1.06	0.00	False

graph graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1910861947368425 > 1.871897771051555. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -58.10619689889136 (T) = (0 -2193310.287413498) / Math.Sqrt((873200832.3731902 / (30)) + (73438595.29571351 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.2053272089634021 = (1819680.3914347664 - 2193310.287413498) / 1819680.3914347664 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7828741866666669 > 1.7299899922746715. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/9/2023 4:10:34 PM, 2/24/2023 8:06:06 PM, 4/9/2023 4:09:45 AM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -26.464286222198 (T) = (0 -1773701.6875421756) / Math.Sqrt((639434490.0717189 / (30)) + (101312671.48405826 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.10158482169796228 = (1610136.280570955 - 1773701.6875421756) / 1610136.280570955 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 529.7141253526436 > 525.1344225815831. IsChangePoint: Marked as a change because one of 2/9/2023 11:32:54 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -20.877261489599945 (T) = (0 -526.9293230603633) / Math.Sqrt((20.888554627616656 / (29)) + (6.555693108802181 / (6))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (6) - 2, .025) and -0.05635318851795437 = (498.819266877621 - 526.9293230603633) / 498.819266877621 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.898642068157034 > 1.7733478402702705. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/24/2023 8:06:06 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -30.290687989427315 (T) = (0 -1888922.753114077) / Math.Sqrt((406471938.86858433 / (31)) + (166090404.26350653 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.11410883674841205 = (1695456.2164922792 - 1888922.753114077) / 1695456.2164922792 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.254283375974026 > 1.9818307222846894. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -39.45240782221396 (T) = (0 -2261357.811213686) / Math.Sqrt((362169379.31319964 / (29)) + (351150422.62502754 / (5))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (5) - 2, .025) and -0.18860003795191213 = (1902538.9020769787 - 2261357.811213686) / 1902538.9020769787 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.762482825734549 > 1.6991908784581102. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/24/2023 8:06:06 PM, 4/5/2023 6:15:03 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -34.78771802222719 (T) = (0 -1776104.3081695803) / Math.Sqrt((388249578.8010897 / (30)) + (50642733.984825514 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09958944149409421 = (1615243.1454382236 - 1776104.3081695803) / 1615243.1454382236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.1920711687737042 > 1.9196957530289886. IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/23/2023 8:54:44 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -51.83593603945759 (T) = (0 -2187038.7038047486) / Math.Sqrt((920699703.4092327 / (30)) + (77923994.51734911 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.18573315906034496 = (1844461.1142846893 - 2187038.7038047486) / 1844461.1142846893 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: False, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.8693208967418546 > 1.8131148503124999. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 2/26/2023 8:26:11 PM, 3/28/2023 3:45:48 AM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -35.73761786202674 (T) = (0 -1881961.011604825) / Math.Sqrt((233964047.12943557 / (30)) + (71730178.5476253 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09217957461024825 = (1723124.1595745971 - 1881961.011604825) / 1723124.1595745971 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf8(Formatted: True, SkipValidation: False, DataSize: 100000) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.296002919312169 > 1.965640625443603. IsChangePoint: Marked as a change because one of 2/10/2023 8:30:00 AM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -44.36657378525827 (T) = (0 -2260178.9357005525) / Math.Sqrt((783892845.186126 / (30)) + (312132850.4534875 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.2099701183193885 = (1867962.6062500386 - 2260178.9357005525) / 1867962.6062500386 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: False, DataSize: 10) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 719.9160305390661 > 711.0953909842768. IsChangePoint: Marked as a change because one of 4/3/2023 12:37:39 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -11.72540500345933 (T) = (0 -722.8185962840666) / Math.Sqrt((64.65715477881133 / (30)) + (39.02846101325802 / (5))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (5) - 2, .025) and -0.053960089331072617 = (685.8121133816605 - 722.8185962840666) / 685.8121133816605 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToStream(Mode%3a%20SourceGen).html>)	371.31 ns	410.72 ns	1.11	0.06	False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToWriter(Mode%3a%20SourceGen).html>)	249.92 ns	289.21 ns	1.16	0.02	False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeObjectProperty(Mode%3a%20SourceGen).html>)	541.62 ns	576.38 ns	1.06	0.01	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToString(Mode%3a%20SourceGen).html>)	351.19 ns	396.33 ns	1.13	0.02	False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(LargeStructWithProperties).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>)	318.11 ns	361.77 ns	1.14	0.01	False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 410.71953339516404 > 393.6568450135976. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -11.989753384429102 (T) = (0 -411.2909671443468) / Math.Sqrt((35.173850795785164 / (30)) + (45.813464440516455 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09471094164164598 = (375.70736849269844 - 411.2909671443468) / 375.70736849269844 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 289.2086397212238 > 264.1382180773349. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -49.57662707378554 (T) = (0 -290.10722420149347) / Math.Sqrt((3.4790888600755308 / (30)) + (2.8637003348500913 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.15157623063214878 = (251.92185847934823 - 290.10722420149347) / 251.92185847934823 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 576.3763269366984 > 564.0916531391246. IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 3/20/2023 10:25:17 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -13.32224624495782 (T) = (0 -582.2120900248946) / Math.Sqrt((63.51127023464445 / (30)) + (43.14860946831349 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.07505251269838538 = (541.5661869051776 - 582.2120900248946) / 541.5661869051776 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 396.3297345307911 > 368.0279664242818. IsChangePoint: Marked as a change because one of 2/28/2023 5:58:27 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -16.499505560208334 (T) = (0 -386.5569202679839) / Math.Sqrt((24.309428825154313 / (30)) + (27.31138000780434 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.10967992780688317 = (348.3499255789514 - 386.5569202679839) / 348.3499255789514 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 361.7655428358695 > 334.87722745816814. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -25.17873809223015 (T) = (0 -362.0234382845542) / Math.Sqrt((13.231054017108272 / (30)) + (13.261410300909137 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.12770893623934668 = (321.02560035732284 - 362.0234382845542) / 321.02560035732284 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_Int64.TryFormat(value%3a%20-9223372036854775808).html>)	19.21 ns	24.92 ns	1.30	0.35	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int64.TryFormat(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 24.917779392076703 > 20.469578444672553. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -24.026130304673806 (T) = (0 -24.516687737415506) / Math.Sqrt((0.16750958984691408 / (30)) + (0.24430931300215136 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.26722340401076994 = (19.346776314121122 - 24.516687737415506) / 19.346776314121122 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Compression.Brotli

Benchmark	Baseline	Test	Test/Base	Edge Detector
[Compress_WithState - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Compression.Brotli.Compress_WithState(level%3a%20Optimal%2c%20file%3a%20%22sum%22).html>)	43.67 ms	47.62 ms	1.09	True
[Compress_WithoutState - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Compression.Brotli.Compress_WithoutState(level%3a%20Optimal%2c%20file%3a%20%22alice29.txt%22).html>)	186.29 ms	203.94 ms	1.09	True
[Compress_WithoutState - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.IO.Compression.Brotli.Compress_WithoutState(level%3a%20Optimal%2c%20file%3a%20%22sum%22).html>)	43.57 ms	47.77 ms	1.10	True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Compression.Brotli*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Compression.Brotli.Compress_WithState(level: Optimal, file: "sum") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.6226491923077 > 45.8411344575. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -275.5452784429092 (T) = (0 -47629261.52669414) / Math.Sqrt((5074514366.276683 / (30)) + (261235871.48336983 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09214522596616591 = (43610740.031902745 - 47629261.52669414) / 43610740.031902745 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.IO.Compression.Brotli.Compress_WithoutState(level: Optimal, file: "alice29.txt") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 203.9378077142857 > 195.74759076384615. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -230.15642737597 (T) = (0 -204114005.97460318) / Math.Sqrt((47797757075.086525 / (30)) + (25809448259.87148 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09477909570068029 = (186443097.7684737 - 204114005.97460318) / 186443097.7684737 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.IO.Compression.Brotli.Compress_WithoutState(level: Optimal, file: "sum") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 47.771719616666665 > 45.75532022541666. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -129.83790275503745 (T) = (0 -47686605.147023804) / Math.Sqrt((4161557229.711356 / (30)) + (5179273197.2822695 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09431156060564166 = (43576808.345725484 - 47686605.147023804) / 43576808.345725484 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<StructRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>)	339.89 ns	362.44 ns	1.07	0.03	False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToWriter(Mode%3a%20Reflection).html>)	269.20 ns	296.95 ns	1.10	0.04	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(StructRecord).SerializeToString(Mode%3a%20Reflection).html>)	363.74 ns	388.59 ns	1.07	0.04	False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;StructRecord&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 362.44406296749435 > 354.5714159028385. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -13.547915021226732 (T) = (0 -362.88011562319997) / Math.Sqrt((39.7268759387784 / (30)) + (6.506948966564297 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.06150712507254802 = (341.8536786537341 - 362.88011562319997) / 341.8536786537341 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 296.95195539646534 > 286.23947832288053. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -26.182360461772696 (T) = (0 -298.61939268954933) / Math.Sqrt((23.26200374971256 / (30)) + (1.4229519715703485 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.09676407017496098 = (272.27313586404426 - 298.61939268954933) / 272.27313586404426 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<StructRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 388.5910713168665 > 381.4157861349997. IsChangePoint: Marked as a change because one of 3/21/2023 7:06:30 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -14.039553410907343 (T) = (0 -392.05869485994225) / Math.Sqrt((41.126930175691555 / (30)) + (11.275276748640525 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.06901356814422167 = (366.7480998772966 - 392.05869485994225) / 366.7480998772966 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[Tan - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Tan.html>)	46.02 μs	48.51 μs	1.05	0.03	False
[Atan2 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Atan2.html>)	64.20 μs	67.92 μs	1.06	0.05	False
[AsinPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.AsinPi.html>)	24.55 μs	26.61 μs	1.08	0.04	False
[AcosPi - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.AcosPi.html>)	31.58 μs	33.60 μs	1.06	0.03	False
[Acos - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Acos.html>)	27.93 μs	30.00 μs	1.07	0.04	False
[Asin - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.MathBenchmarks.Single.Asin.html>)	21.42 μs	23.39 μs	1.09	0.05	False

graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.MathBenchmarks.Single.Tan ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 48.51426324174406 > 48.333473092830886. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -55.333595981743436 (T) = (0 -48540.5120780177) / Math.Sqrt((51796.31317410045 / (30)) + (1480.5313272337867 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.05333955413660209 = (46082.492475852414 - 48540.5120780177) / 46082.492475852414 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.MathBenchmarks.Single.Atan2 ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 67.91729910117057 > 67.52383261752746. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -100.63919655706607 (T) = (0 -67985.8250957276) / Math.Sqrt((11173.80890782264 / (30)) + (6703.390646464205 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.05706788511403592 = (64315.47685170033 - 67985.8250957276) / 64315.47685170033 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.MathBenchmarks.Single.AsinPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 26.610528181689336 > 25.784699381585106. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -85.9534120408644 (T) = (0 -26618.040037311355) / Math.Sqrt((13567.614096044717 / (30)) + (739.6910573142018 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.08257169900384773 = (24587.784866170558 - 26618.040037311355) / 24587.784866170558 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.MathBenchmarks.Single.AcosPi ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 33.598266845493555 > 33.153559476356456. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/11/2023 4:36:40 PM, 4/12/2023 9:47:42 AM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -4.924587238866199 (T) = (0 -34080.10456946933) / Math.Sqrt((8316.373729737992 / (29)) + (1755940.685180966 / (7))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (7) - 2, .025) and -0.07806722896177516 = (31612.225707194466 - 34080.10456946933) / 31612.225707194466 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.MathBenchmarks.Single.Acos ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 29.9985994921625 > 29.32524347265625. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -13.981138707496118 (T) = (0 -30025.954179964974) / Math.Sqrt((589511.2521937377 / (30)) + (511.1693576910824 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.06999246398485204 = (28061.83706018144 - 30025.954179964974) / 28061.83706018144 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.MathBenchmarks.Single.Asin ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 23.389157448928746 > 22.48621118569401. IsChangePoint: Marked as a change because one of 4/11/2023 10:53:42 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -8.97810341794848 (T) = (0 -23185.003756430542) / Math.Sqrt((468.80564352178015 / (27)) + (387327.74917029054 / (10))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (10) - 2, .025) and -0.0825181014957419 = (21417.659182230072 - 23185.003756430542) / 21417.659182230072 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>)	341.97 ns	372.58 ns	1.09	0.06	False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToWriter(Mode%3a%20Reflection).html>)	270.32 ns	292.15 ns	1.08	0.09	False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Serialization.Tests.WriteJson(ClassRecord).SerializeToString(Mode%3a%20Reflection).html>)	365.72 ns	386.15 ns	1.06	0.03	False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ClassRecord&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 372.5819753029107 > 353.95263909633377. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -6.361776943346956 (T) = (0 -374.02895584759284) / Math.Sqrt((43.67648382975904 / (31)) + (150.18535771900207 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.09584066174272436 = (341.3169166882086 - 374.02895584759284) / 341.3169166882086 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 292.1538843373581 > 283.77123482338027. IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -7.048548410161493 (T) = (0 -294.51347642743957) / Math.Sqrt((191.21531783423376 / (31)) + (7.228726619062785 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.06950227940398099 = (275.3743326209346 - 294.51347642743957) / 275.3743326209346 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<ClassRecord>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 386.15050066101935 > 385.31118300491. IsChangePoint: Marked as a change because one of 3/20/2023 10:25:17 PM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -6.62167673152512 (T) = (0 -393.4978580799622) / Math.Sqrt((37.797812732398874 / (30)) + (87.05565281982838 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.07160886067122728 = (367.20287832771896 - 393.4978580799622) / 367.20287832771896 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_UInt32.ToString(value%3a%204294967295).html>)	12.62 ns	13.97 ns	1.11	0.02	False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_UInt32.ToString(value%3a%2012345).html>)	10.26 ns	11.41 ns	1.11	0.04	False
[TryFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Tests.Perf_UInt32.TryFormat(value%3a%204294967295).html>)	5.01 ns	6.01 ns	1.20	0.03	True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt32.ToString(value: 4294967295) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 13.966395588652512 > 13.560038416359673. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -8.666155356357653 (T) = (0 -14.145776003283913) / Math.Sqrt((0.03221905078477859 / (30)) + (0.1300870123159681 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.10182438642764954 = (12.838503283765162 - 14.145776003283913) / 12.838503283765162 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_UInt32.ToString(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 11.41137023444223 > 10.966271915283356. IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 3/10/2023 8:06:53 PM, 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -10.381420235010925 (T) = (0 -11.252555930884787) / Math.Sqrt((0.012253794304979021 / (30)) + (0.03306725268600964 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.07640655558205399 = (10.453815867741628 - 11.252555930884787) / 10.453815867741628 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_UInt32.TryFormat(value: 4294967295) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.010211868992781 > 5.254095661393162. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -18.86075495891155 (T) = (0 -5.942634930846046) / Math.Sqrt((0.005288486838997106 / (30)) + (0.016092602534209015 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.20436038792212147 = (4.934266346221211 - 5.942634930846046) / 4.934266346221211 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>)	46.80 ms	68.20 ms	1.46	0.04	True
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20True%2c%20Escaped%3a%20AllEscaped).html>)	46.87 ms	68.23 ms	1.46	0.12	True
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20False%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>)	46.54 ms	68.35 ms	1.47	0.06	True
[WriteStringsUtf8 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted%3a%20True%2c%20SkipValidation%3a%20False%2c%20Escaped%3a%20AllEscaped).html>)	47.01 ms	69.13 ms	1.47	0.10	True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.19994863333334 > 49.26532368125. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -100.53199799827438 (T) = (0 -68345053.0827381) / Math.Sqrt((1046841813799.2554 / (30)) + (53394304979.30654 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.4447206786913732 = (47306759.07860957 - 68345053.0827381) / 47306759.07860957 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: True, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.2311915 > 49.69782721125001. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -85.95693762650454 (T) = (0 -68320309.42557998) / Math.Sqrt((1169331405623.2295 / (30)) + (105819209549.52011 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.4272667982785732 = (47867931.56540958 - 68320309.42557998) / 47867931.56540958 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 68.34721616071428 > 49.02595604350962. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -41.887601599208075 (T) = (0 -68578710.46646826) / Math.Sqrt((1402526989086.0103 / (30)) + (1226798629397.3135 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.4412142519480071 = (47583980.226239316 - 68578710.46646826) / 47583980.226239316 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: True, SkipValidation: False, Escaped: AllEscaped) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 69.13307721666666 > 49.48338324461538. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -68.37349157090426 (T) = (0 -68892223.53388278) / Math.Sqrt((1633047269440.5789 / (30)) + (192950113651.84363 / (5))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (5) - 2, .025) and -0.43410987101212617 = (48038316.259033866 - 68892223.53388278) / 48038316.259033866 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

performanceautofiler[bot] commented 1 year ago

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).GetHashCodeBenchmark.html>)	7.52 ns	8.55 ns	1.14	0.14	False
[SquareRootBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(UInt64).SquareRootBenchmark.html>)	6.67 ns	7.77 ns	1.17	0.03	True

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 8.548883624772696 > 7.784270405766154. IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 3/27/2023 11:25:56 PM, 3/28/2023 8:13:28 AM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -3.9030052680096623 (T) = (0 -8.604064321611782) / Math.Sqrt((1.7050701281295564 / (31)) + (0.01359647145042256 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.12177512837923633 = (7.670043758273648 - 8.604064321611782) / 7.670043758273648 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.SquareRootBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 7.773890537150653 > 6.999588574619414. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -114.72169874622048 (T) = (0 -7.782419648958388) / Math.Sqrt((0.0020619044772005536 / (30)) + (0.00014260004563110734 / (6))) is less than -2.0322445093148245 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (6) - 2, .025) and -0.1651932480676292 = (6.679080626209299 - 7.782419648958388) / 6.679080626209299 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	f4701e21bcf9e04ba9ed1ffc4785889d36119242
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[GroupBy - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Linq.Tests.Perf_Enumerable.GroupBy(input%3a%20IEnumerable).html>)	2.15 μs	2.57 μs	1.19	0.00	True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.GroupBy(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 2.567518692724244 > 2.2629801019639744. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -37.918191625801136 (T) = (0 -2545.785917976425) / Math.Sqrt((152.7388112503231 / (29)) + (476.7943101518483 / (5))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (5) - 2, .025) and -0.1756548151117479 = (2165.419547687936 - 2545.785917976425) / 2165.419547687936 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	e7ed417c2bf4a24a0694a387f33ba843a2318fdc
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlyMemory<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[Pin - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Memory.ReadOnlyMemory(Char).Pin(Size%3a%20512).html>)	32.46 ns	34.42 ns	1.06	0.09	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlyMemory&lt;Char&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.ReadOnlyMemory<Char>.Pin(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 34.41680158243167 > 34.16487049505954. IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -7.0277531617541005 (T) = (0 -34.681156285866564) / Math.Sqrt((0.3055724499246111 / (29)) + (0.400919158726846 / (6))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (6) - 2, .025) and -0.05972635749846771 = (32.72652042715349 - 34.681156285866564) / 32.72652042715349 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	c1b7a9feb6f3b4d9ca27dc4f74d8260e4edb73e8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04/System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int64).GetHashCodeBenchmark.html>)	7.41 ns	9.03 ns	1.22	0.08	False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

### Payloads [Baseline]() [Compare]() ### Histogram #### System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.025873244830283 > 7.769292847910753. IsChangePoint: Marked as a change because one of 2/9/2023 11:32:54 PM, 2/14/2023 11:21:41 AM, 2/20/2023 10:31:48 AM, 4/12/2023 8:39:35 PM, 4/18/2023 4:36:42 PM falls between 4/9/2023 11:04:55 PM and 4/18/2023 4:36:42 PM. IsRegressionStdDev: Marked as regression because -14.668296657394158 (T) = (0 -8.622655432631268) / Math.Sqrt((0.0005461483029139432 / (31)) + (0.040546822163966156 / (6))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (6) - 2, .025) and -0.1628250355492953 = (7.4152646950519925 - 8.622655432631268) / 7.4152646950519925 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

dotnet / perf-autofiling-issues