search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 38 Regressions on 4/9/2023 11:24:15 AM #15740

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Append_ValueTypes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes.html>) 6.30 μs 6.85 μs 1.09 0.09 False
[Append_ValueTypes_Interpolated - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated.html>) 8.45 μs 9.69 μs 1.15 0.12 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 6.845737628565119 > 6.409512357608278. IsChangePoint: Marked as a change because one of 2/12/2023 3:00:19 AM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -11.650297722022461 (T) = (0 -6954.189314989111) / Math.Sqrt((19648.708733229687 / (43)) + (34073.82435458719 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.13000695139437102 = (6154.111978167918 - 6954.189314989111) / 6154.111978167918 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Tests.Perf_StringBuilder.Append_ValueTypes_Interpolated ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 9.692109014917694 > 8.870198687098206. IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -39.397604218814955 (T) = (0 -9742.297103665582) / Math.Sqrt((28780.18582165279 / (43)) + (3075.7982203304246 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.15111293739164108 = (8463.372087313252 - 9742.297103665582) / 8463.372087313252 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 1.45 μs 1.60 μs 1.11 0.25 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 743.14 ns 884.26 ns 1.19 0.20 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20Reflection).html>) 564.19 ns 699.97 ns 1.24 0.19 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 773.45 ns 895.81 ns 1.16 0.23 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20Reflection).html>) 943.74 ns 1.02 μs 1.08 0.25 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20SourceGen).html>) 596.49 ns 656.28 ns 1.10 0.20 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20SourceGen).html>) 889.06 ns 1.04 μs 1.17 0.22 False

graph_1.png>) graph_2.png>) graph_3.png>) graph_4.png>) graph_5.png>) graph_6.png>) Test Report.html>)

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6033726002028117 > 1.5696686159442366. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -7.126789045032063 (T) = (0 -1677.5116093092888) / Math.Sqrt((1787.0238810917895 / (42)) + (3469.6964297494897 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.10218963703757308 = (1521.980930448635 - 1677.5116093092888) / 1521.980930448635 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 884.260451472327 > 784.6956055090557. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -18.141435783112605 (T) = (0 -884.3718620252359) / Math.Sqrt((493.35994200494923 / (43)) + (312.3531094186463 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.17068633534156816 = (755.4302423519832 - 884.3718620252359) / 755.4302423519832 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 699.9730267259536 > 596.0854349810462. IsChangePoint: Marked as a change because one of 3/13/2023 8:52:40 AM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -15.94312465013546 (T) = (0 -669.8157709488077) / Math.Sqrt((349.4172701304029 / (43)) + (320.8793133395058 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.19805183021287465 = (559.0874735609661 - 669.8157709488077) / 559.0874735609661 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 895.8106298185613 > 805.9544086248478. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -10.076381235959346 (T) = (0 -883.5797060306238) / Math.Sqrt((290.62529696233787 / (43)) + (1221.7855120181046 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.1682478720451732 = (756.3289668003418 - 883.5797060306238) / 756.3289668003418 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.020953510086044 > 960.3639619819999. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -15.137351829487514 (T) = (0 -1039.4775770263705) / Math.Sqrt((939.6907994466882 / (43)) + (321.75023756590133 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.13626545555240005 = (914.8193073607296 - 1039.4775770263705) / 914.8193073607296 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 656.2821395411953 > 596.9572439283145. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -20.25929075714919 (T) = (0 -665.7558939639472) / Math.Sqrt((254.69592280282856 / (43)) + (162.31236365080755 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.19643789172186193 = (556.448352706232 - 665.7558939639472) / 556.448352706232 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0379509105271443 > 933.6215003256035. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -11.806141000660999 (T) = (0 -1061.3698151491956) / Math.Sqrt((802.894219685747 / (43)) + (1234.729182493035 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.17138850624652813 = (906.0783928554458 - 1061.3698151491956) / 906.0783928554458 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_DateTimes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 8.46 ms 15.23 ms 1.80 0.09 True
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 8.37 ms 15.61 ms 1.86 0.06 True
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 7.27 ms 15.52 ms 2.14 0.11 True
[WriteDateTimes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 7.36 ms 16.74 ms 2.27 0.11 True

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_DateTimes*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.232602665178572 > 8.8966080478125. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -20.321759520324875 (T) = (0 -15917815.69438969) / Math.Sqrt((101391066844.01012 / (42)) + (1016551751001.3102 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.8497722423233546 = (8605284.115625264 - 15917815.69438969) / 8605284.115625264 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.606166670833332 > 8.826791964843752. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -28.007805150674194 (T) = (0 -16167077.27452877) / Math.Sqrt((14463057281.009233 / (43)) + (610502186738.8705 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.9216884232308075 = (8412954.503492368 - 16167077.27452877) / 8412954.503492368 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 15.517875022321428 > 7.70143438125. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -40.0803292901393 (T) = (0 -15362533.35529285) / Math.Sqrt((37287439335.00616 / (42)) + (322409989469.74115 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -1.125361822656728 = (7228196.719977541 - 15362533.35529285) / 7228196.719977541 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 16.738729591836737 > 7.5158398423076935. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -35.783419745786205 (T) = (0 -15315042.906856386) / Math.Sqrt((61407129919.42048 / (42)) + (408957813519.81976 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -1.1541216047098208 = (7109646.397571625 - 15315042.906856386) / 7109646.397571625 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) 32.76 ns 143.95 ns 4.39 0.45 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%209223372036854775807).html>) 33.22 ns 139.12 ns 4.19 0.41 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%20-9223372036854775808).html>) 36.90 ns 169.17 ns 4.58 0.40 False

graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 143.95497334275439 > 35.412342602779404. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -97.84853101811268 (T) = (0 -147.24701806844163) / Math.Sqrt((6.301747962894654 / (42)) + (9.295621810125512 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -3.1868365591848216 = (35.16903895983718 - 147.24701806844163) / 35.16903895983718 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 139.12078431727303 > 34.36084923962718. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -144.76842350601208 (T) = (0 -140.65257869880358) / Math.Sqrt((5.94395574219608 / (43)) + (3.2444189305380804 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -3.1493036024139025 = (33.89787592717424 - 140.65257869880358) / 33.89787592717424 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 169.16630854439265 > 35.792598546829836. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -27.70409371678875 (T) = (0 -142.17819012624625) / Math.Sqrt((3.246279537751629 / (43)) + (120.56562348897737 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -3.1380097298467113 = (34.35907583801479 - 142.17819012624625) / 34.35907583801479 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22o%22).html>) 130.53 ns 176.35 ns 1.35 0.24 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22s%22).html>) 370.78 ns 610.28 ns 1.65 0.13 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20null).html>) 360.58 ns 594.18 ns 1.65 0.16 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Tests.Perf_DateTime.ToString(format%3a%20%22G%22).html>) 358.66 ns 579.81 ns 1.62 0.15 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_DateTime.ToString(format: "o") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 176.35072100136622 > 136.5719478509588. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -28.935159052823906 (T) = (0 -178.42441667185994) / Math.Sqrt((26.735841185961704 / (43)) + (19.504432413232657 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.3960122253959396 = (127.81006743780834 - 178.42441667185994) / 127.81006743780834 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Tests.Perf_DateTime.ToString(format: "s") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 610.2778170224747 > 389.66698580220293. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -17.765278436252178 (T) = (0 -567.9044890816053) / Math.Sqrt((150.855263664341 / (42)) + (895.3739359662031 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.5065049874447601 = (376.96821040390284 - 567.9044890816053) / 376.96821040390284 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Tests.Perf_DateTime.ToString(format: null) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 594.1806829059494 > 377.39721273111485. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -27.642079011066222 (T) = (0 -600.8701035130399) / Math.Sqrt((151.86787165916476 / (43)) + (580.0321009110728 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.6698477703668826 = (359.8352581451318 - 600.8701035130399) / 359.8352581451318 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Tests.Perf_DateTime.ToString(format: "G") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 579.8074739088078 > 363.7887397283544. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -27.04407877343501 (T) = (0 -598.575252166127) / Math.Sqrt((123.23113972028878 / (43)) + (622.5515888155318 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.6830335661701122 = (355.65259314954517 - 598.575252166127) / 355.65259314954517 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 889.05 μs 1.16 ms 1.30 0.08 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToWriter(Mode%3a%20Reflection).html>) 1.41 ms 1.72 ms 1.22 0.08 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20Reflection).html>) 1.44 ms 1.76 ms 1.23 0.05 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 854.43 μs 1.10 ms 1.28 0.11 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 1.44 ms 1.74 ms 1.21 0.06 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20SourceGen).html>) 917.53 μs 1.17 ms 1.27 0.10 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeObjectProperty(Mode%3a%20Reflection).html>) 1.47 ms 1.75 ms 1.19 0.09 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(MyEventsListerViewModel).SerializeToString(Mode%3a%20SourceGen).html>) 928.36 μs 1.22 ms 1.32 0.12 False

graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1593294849358977 > 935.54041484375. IsChangePoint: Marked as a change because one of 2/16/2023 3:28:45 AM, 3/10/2023 4:14:40 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -54.49145643838688 (T) = (0 -1146674.4874828297) / Math.Sqrt((498266766.82401186 / (43)) + (80003993.02717009 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.28336453429719904 = (893490.8647064772 - 1146674.4874828297) / 893490.8647064772 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7162342578703706 > 1.4829773012784087. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -31.850399402584543 (T) = (0 -1688859.7521420305) / Math.Sqrt((1117112246.5810547 / (42)) + (430297621.2485525 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.2034948073343179 = (1403296.2517576392 - 1688859.7521420305) / 1403296.2517576392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.764974696428571 > 1.548973780778846. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -17.594688621154177 (T) = (0 -1792173.5302770543) / Math.Sqrt((622096772.6900558 / (42)) + (2588988922.1747136 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.22041861519251738 = (1468490.817795617 - 1792173.5302770543) / 1468490.817795617 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.0969574398809527 > 901.3729586266448. IsChangePoint: Marked as a change because one of 3/10/2023 11:41:11 AM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -25.793097502294057 (T) = (0 -1119286.0432728126) / Math.Sqrt((418743005.78988874 / (43)) + (621068929.4863354 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.29645160183200586 = (863345.7984016974 - 1119286.0432728126) / 863345.7984016974 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.7445064828703702 > 1.5106449220028406. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -21.736731935296923 (T) = (0 -1758529.536543072) / Math.Sqrt((764249190.579451 / (43)) + (1670878185.9733398 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.22862616136269096 = (1431297.4864483236 - 1758529.536543072) / 1431297.4864483236 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1693092935897438 > 960.3113865820311. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 3/10/2023 4:14:40 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -35.43976188208269 (T) = (0 -1185219.1686541324) / Math.Sqrt((307815688.6058221 / (43)) + (380187820.1844145 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.28388326367679156 = (923151.8177594243 - 1185219.1686541324) / 923151.8177594243 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.747175462037037 > 1.554131670706731. IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -19.026915485400853 (T) = (0 -1761561.3764147067) / Math.Sqrt((381766522.5330156 / (42)) + (1793045981.4603171 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.19753394388143808 = (1470990.7684997614 - 1761561.3764147067) / 1470990.7684997614 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<MyEventsListerViewModel>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.2218868048611111 > 965.5757336914063. IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -37.23749791343162 (T) = (0 -1187560.780715812) / Math.Sqrt((439787795.285117 / (42)) + (325027385.48840934 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.28890421195906957 = (921372.4105306315 - 1187560.780715812) / 921372.4105306315 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 33.74 μs 35.94 μs 1.07 0.03 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Text.Json.Serialization.Tests.WriteJson(IndexViewModel).SerializeToWriter(Mode%3a%20SourceGen).html>) 27.52 μs 32.51 μs 1.18 0.04 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 35.94474387423547 > 33.46590119859738. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 3/1/2023 3:09:21 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -21.79088070033398 (T) = (0 -36306.88545912535) / Math.Sqrt((248191.0850875688 / (42)) + (279895.72386670613 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.1381414155857684 = (31900.153146117827 - 36306.88545912535) / 31900.153146117827 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 32.50797598812015 > 29.002805014581433. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 3/1/2023 3:09:21 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -23.924574494466015 (T) = (0 -32205.20385495564) / Math.Sqrt((123359.35798565281 / (43)) + (262021.3156705044 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.163072608658262 = (27689.762113913115 - 32205.20385495564) / 27689.762113913115 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 356.11 ns 566.17 ns 1.59 0.10 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 349.36 ns 565.71 ns 1.62 0.09 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 358.67 ns 554.91 ns 1.55 0.08 True
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20da).html>) 344.73 ns 566.45 ns 1.64 0.12 True

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 566.1664666717122 > 369.19276072376414. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -34.74832533092831 (T) = (0 -585.1597735683579) / Math.Sqrt((39.81464781494948 / (42)) + (358.3305198442773 / (8))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (8) - 2, .025) and -0.6711498184530165 = (350.15398805478753 - 585.1597735683579) / 350.15398805478753 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 565.7073154018273 > 367.85619242284764. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -80.97188928082883 (T) = (0 -569.8443059311339) / Math.Sqrt((48.80038574832103 / (43)) + (48.14050357966649 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.6129534181544906 = (353.292475478392 - 569.8443059311339) / 353.292475478392 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 554.9068956724464 > 364.66230665573227. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -52.155757611447925 (T) = (0 -572.0325895330441) / Math.Sqrt((160.86267299196928 / (43)) + (111.58322496242347 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.621982136037345 = (352.67502447996964 - 572.0325895330441) / 352.67502447996964 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 566.4469770637679 > 371.1137097895988. IsChangePoint: Marked as a change because one of 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -21.25151465451916 (T) = (0 -587.6412895726675) / Math.Sqrt((90.01340326392385 / (43)) + (967.5727003778198 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.6698513120133229 = (351.9123441380863 - 587.6412895726675) / 351.9123441380863 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(MyEventsListerViewModel).SystemTextJsonSourceGen.html>) 1.30 ms 1.60 ms 1.23 0.07 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToStream(MyEventsListerViewModel).SystemTextJsonReflection.html>) 1.38 ms 1.68 ms 1.22 0.06 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToStream&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6045782064102563 > 1.3721657819213418. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -19.234449545203493 (T) = (0 -1601269.2856201446) / Math.Sqrt((1284416405.5164037 / (41)) + (1421352772.449987 / (8))) is less than -2.011740513728388 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (8) - 2, .025) and -0.2101467941413823 = (1323202.5183823006 - 1601269.2856201446) / 1323202.5183823006 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### MicroBenchmarks.Serializers.Json_ToStream<MyEventsListerViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.6838825419501133 > 1.4672714498997175. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -25.425466489744373 (T) = (0 -1690480.7134120748) / Math.Sqrt((531703895.71895164 / (43)) + (910712443.4010632 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.2033181718482459 = (1404849.3182942362 - 1690480.7134120748) / 1404849.3182942362 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture x64
OS ubuntu 18.04
Queue TigerUbuntu
Baseline 1411364699b5784040e86f76cb3db8200f6a2c8c
Compare 86b48d7c6f081c12dcc9c048fb53de1b78c9966f
Diff Diff
Configs AOT:true, CompilationMode:wasm, RunKind:micro

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonSourceGen - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonSourceGen.html>) 907.61 μs 1.16 ms 1.28 0.08 False
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_AOT=true_CompilationMode=wasm_RunKind=micro/MicroBenchmarks.Serializers.Json_ToString(MyEventsListerViewModel).SystemTextJsonReflection.html>) 1.46 ms 1.84 ms 1.26 0.07 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_SourceGen_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.1594910253205126 > 951.2720519140626. IsChangePoint: Marked as a change because one of 2/12/2023 11:53:55 PM, 3/10/2023 8:06:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -35.80524198497207 (T) = (0 -1168676.8059380723) / Math.Sqrt((373949062.6043899 / (43)) + (342213006.19311845 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.2817348812973155 = (911792.9323692815 - 1168676.8059380723) / 911792.9323692815 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small. IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline. IsRegressionWindowed: Marked as regression because 1.841139141402715 > 1.527581492286383. IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/22/2023 10:50:22 PM, 3/23/2023 11:51:53 PM, 4/9/2023 4:09:45 AM, 4/11/2023 5:52:15 AM falls between 3/30/2023 4:24:24 AM and 4/11/2023 5:52:15 AM. IsRegressionStdDev: Marked as regression because -16.28395472323799 (T) = (0 -1784451.3730962027) / Math.Sqrt((8725171752.854292 / (43)) + (1198453249.0073917 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.21433813029392135 = (1469484.7576467765 - 1784451.3730962027) / 1469484.7576467765 is less than -0.05. IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
radekdoulik commented 1 year ago

this is also https://github.com/dotnet/runtime/pull/84469

stephentoub commented 1 year ago

How many issues do we need tracking the same thing? :-)