search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 1074 Improvements on 5/19/2023 1:23:34 PM #18007

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToStream(Mode%3a%20SourceGen).html>) 335.90 ns 279.09 ns 0.83 0.28 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20Reflection).html>) 130.10 ns 103.97 ns 0.80 0.01 False
[SerializeToWriter - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToWriter(Mode%3a%20SourceGen).html>) 129.51 ns 104.49 ns 0.81 0.01 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20SourceGen).html>) 260.72 ns 215.65 ns 0.83 0.06 False
[SerializeToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToString(Mode%3a%20Reflection).html>) 258.46 ns 209.46 ns 0.81 0.11 False
[SerializeObjectProperty - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeObjectProperty(Mode%3a%20Reflection).html>) 434.60 ns 324.58 ns 0.75 0.14 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20Reflection).html>) 230.60 ns 190.64 ns 0.83 0.18 False
[SerializeToUtf8Bytes - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToUtf8Bytes(Mode%3a%20SourceGen).html>) 234.07 ns 183.40 ns 0.78 0.12 False
[SerializeToStream - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.WriteJson(Nullable(DateTimeOffset)).SerializeToStream(Mode%3a%20Reflection).html>) 358.69 ns 262.51 ns 0.73 0.31 False

graph graph graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 279.09143881040376 < 320.0221502415687. IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 11.696631139380509 (T) = (0 -276.9024655181131) / Math.Sqrt((120.16467472348238 / (299)) + (111.90983099926451 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1677946394249869 = (332.7333355877293 - 276.9024655181131) / 332.7333355877293 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 103.97272678406881 < 124.07791520586807. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 47.14739852360552 (T) = (0 -105.33882887285809) / Math.Sqrt((14.908819269014339 / (299)) + (1.2955692465824875 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19922440408078473 = (131.54600291226282 - 105.33882887285809) / 131.54600291226282 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToWriter(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 104.48953026038403 < 124.85694415645942. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 42.903001085906745 (T) = (0 -104.67156088466675) / Math.Sqrt((15.58186179443758 / (299)) + (1.6589345255863914 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.20252743667554743 = (131.25412170710757 - 104.67156088466675) / 131.25412170710757 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 215.6479588956057 < 246.68632723565239. IsChangePoint: Marked as a change because one of 3/18/2023 7:37:20 PM, 4/19/2023 4:06:12 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 13.809855799607552 (T) = (0 -215.82734382130428) / Math.Sqrt((45.056334976473686 / (299)) + (39.46155562429356 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1535928707074292 = (254.99235102342925 - 215.82734382130428) / 254.99235102342925 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToString(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 209.45587609743214 < 247.1918425301113. IsChangePoint: Marked as a change because one of 3/13/2023 2:14:21 PM, 4/19/2023 10:34:24 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 15.378795851963348 (T) = (0 -209.7445857596024) / Math.Sqrt((68.94923110627033 / (299)) + (35.94480077659512 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.18206979126234699 = (256.43335277139386 - 209.7445857596024) / 256.43335277139386 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeObjectProperty(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 324.5807094203274 < 418.6777867864643. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/17/2023 2:42:45 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 17.386756631006364 (T) = (0 -334.7282876897904) / Math.Sqrt((259.663918327795 / (299)) + (90.77189383098927 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1847049081208162 = (410.5609012293586 - 334.7282876897904) / 410.5609012293586 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 190.6421765984588 < 216.85089695656274. IsChangePoint: Marked as a change because one of 2/15/2023 3:19:23 AM, 4/19/2023 10:34:24 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 16.61163446083379 (T) = (0 -185.49409944509563) / Math.Sqrt((90.3335555790912 / (299)) + (28.83038371027705 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18073301997363533 = (226.41471457707996 - 185.49409944509563) / 226.41471457707996 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToUtf8Bytes(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 183.3993808477538 < 218.02675068172942. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 22.108761916638404 (T) = (0 -181.99932075930477) / Math.Sqrt((68.10719887346097 / (299)) + (18.40513648215543 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19365829014061167 = (225.70991743815685 - 181.99932075930477) / 225.70991743815685 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Text.Json.Serialization.Tests.WriteJson<Nullable<DateTimeOffset>>.SerializeToStream(Mode: Reflection) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 262.5081076074347 < 337.16563786274526. IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 5/10/2023 2:36:50 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 15.158051169441734 (T) = (0 -269.24501483894085) / Math.Sqrt((166.3420367624602 / (299)) + (89.33444822070003 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.19461704419015702 = (334.30681999994005 - 269.24501483894085) / 334.30681999994005 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Array_ComparerClass - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.Sort(String).Array_ComparerClass(Size%3a%20512).html>) 376.40 μs 352.67 μs 0.94 0.02 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.Sort<String>.Array_ComparerClass(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 352.67341473333335 < 359.27343583166663. IsChangePoint: Marked as a change because one of 3/21/2023 3:33:39 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 23.270706732195514 (T) = (0 -353538.42424333334) / Math.Sqrt((38635200.19260948 / (299)) + (2082274.060318605 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.05038538604767105 = (372296.7391707402 - 353538.42424333334) / 372296.7391707402 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<String, String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FrozenDictionaryOptimized - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.TryGetValueFalse(String%2c%20String).FrozenDictionaryOptimized(Size%3a%20512).html>) 12.08 μs 9.11 μs 0.75 0.01 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;String, String&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<String, String>.FrozenDictionaryOptimized(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.114091899698781 < 11.398519508479035. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 110.15577964713843 (T) = (0 -9182.850791359951) / Math.Sqrt((3698.606949412781 / (177)) + (3132.317177449458 / (5))) is greater than 1.9732308230712676 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (177) + (5) - 2, .975) and 0.23384132113168393 = (11985.573021144695 - 9182.850791359951) / 11985.573021144695 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Secant

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Benchstone.BenchF.Secant.Test.html>) 230.23 ms 165.16 ms 0.72 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Secant*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Secant.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 165.15906533333333 < 218.82426505166663. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 974.2791271436296 (T) = (0 -165053580.67333335) / Math.Sqrt((151524320875.6584 / (299)) + (20023492737.82053 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.28391203583665675 = (230493443.45031303 - 165053580.67333335) / 230493443.45031303 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.MDBenchI.MDPuzzle

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Benchstone.MDBenchI.MDPuzzle.Test.html>) 705.47 ms 609.58 ms 0.86 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDPuzzle*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.MDBenchI.MDPuzzle.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 609.5847185333333 < 670.1766192210713. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 252.77846038359579 (T) = (0 -609222084.4371428) / Math.Sqrt((22732073659040.863 / (299)) + (394528804540.9861 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14039037354594294 = (708719476.4793662 - 609222084.4371428) / 708719476.4793662 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DataContractSerializerBinaryXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/MicroBenchmarks.Serializers.Xml_FromStream(XmlElement).DataContractSerializerBinaryXml.html>) 1.21 μs 920.19 ns 0.76 0.55 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;XmlElement&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>.DataContractSerializer_BinaryXml_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 920.1853934082143 < 1.1247274740060935. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 9.72939633594677 (T) = (0 -914.6541140242505) / Math.Sqrt((5017.605162698484 / (217)) + (2795.7354924819324 / (5))) is greater than 1.970805592384696 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (217) + (5) - 2, .975) and 0.20425212634302667 = (1149.4270287155484 - 914.6541140242505) / 1149.4270287155484 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in PerfLabTests.DelegatePerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DelegateInvoke - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.DelegatePerf.DelegateInvoke.html>) 667.28 μs 133.46 μs 0.20 0.38 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.DelegatePerf*'
### Payloads [Baseline]() [Compare]() ### Histogram #### PerfLabTests.DelegatePerf.DelegateInvoke ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 133.46268721751412 < 507.123029095219. IsChangePoint: Marked as a change because one of 5/16/2023 12:30:38 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 41.396108164499154 (T) = (0 -133460.21106638416) / Math.Sqrt((13204548319.721014 / (299)) + (5.272546328349301 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.6733376879362943 = (408557.11276652187 - 133460.21106638416) / 408557.11276652187 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetDocumentElement - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement.html>) 7.69 ns 5.24 ns 0.68 0.70 False
[LoadXml - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.LoadXml.html>) 3.84 μs 2.86 μs 0.74 0.41 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument*'
### Payloads [Baseline]() [Compare]() ### Histogram #### XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.GetDocumentElement ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 5.238107712930679 < 7.362028862802388. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 8.5235811466465 (T) = (0 -4.716221953541931) / Math.Sqrt((0.6089797522130164 / (299)) + (0.25770977015516316 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.29494778209836553 = (6.689181075946797 - 4.716221953541931) / 6.689181075946797 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### XmlDocumentTests.XmlDocumentTests.Perf_XmlDocument.LoadXml ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 2.8562465389172442 < 3.4908347093988565. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 8.329458767346452 (T) = (0 -2990.3727810114165) / Math.Sqrt((21821.438932186553 / (299)) + (32364.046092928125 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.1839118033621371 = (3664.276475669194 - 2990.3727810114165) / 3664.276475669194 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.Regula

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Benchstone.BenchF.Regula.Test.html>) 301.40 ms 260.30 ms 0.86 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Regula*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.Regula.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 260.30110893333335 < 279.8325578083333. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 374.41319769409506 (T) = (0 -260160385.64000002) / Math.Sqrt((1477371999500.6614 / (299)) + (21669831816.28965 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12172940843535583 = (296218942.24708444 - 260160385.64000002) / 296218942.24708444 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/MicroBenchmarks.Serializers.Json_FromStream(CollectionsOfPrimitives).SystemTextJsonReflection.html>) 399.46 μs 298.55 μs 0.75 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromStream&lt;CollectionsOfPrimitives&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromStream<CollectionsOfPrimitives>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 298.5535707343907 < 376.6807027265672. IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 133.68814881809587 (T) = (0 -298907.12784001825) / Math.Sqrt((40687163.384852976 / (299)) + (1666661.8199546011 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23455583717457165 = (390501.5445368139 - 298907.12784001825) / 390501.5445368139 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_FromString<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/MicroBenchmarks.Serializers.Json_FromString(Location).SystemTextJsonReflection.html>) 1.63 μs 1.21 μs 0.74 0.05 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_FromString&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_FromString<Location>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.2085445847610405 < 1.5371659696516353. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 66.00260588076726 (T) = (0 -1220.5837780270099) / Math.Sqrt((1112.2501489079825 / (299)) + (150.82391353151397 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.23941147836813503 = (1604.788585828526 - 1220.5837780270099) / 1604.788585828526 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[TryParse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_UInt16.TryParse(value%3a%20%220%22).html>) 11.58 ns 9.83 ns 0.85 0.21 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_UInt16.ToString(value%3a%200).html>) 2.53 ns 1.37 ns 0.54 0.64 False

graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_UInt16.TryParse(value: "0") ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 9.825872016625334 < 11.027804557934012. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 4.627281298779165 (T) = (0 -9.367938709649955) / Math.Sqrt((0.8720861044233882 / (299)) + (0.10456665851663209 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0708484733648134 = (10.082250785913088 - 9.367938709649955) / 10.082250785913088 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms #### System.Tests.Perf_UInt16.ToString(value: 0) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.370942968642953 < 2.341984764909961. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/24/2023 9:01:57 AM, 3/2/2023 11:15:04 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 12.797601390474407 (T) = (0 -1.4208977112157295) / Math.Sqrt((0.09231809288995224 / (299)) + (0.02568324345216837 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.3992664059696277 = (2.365270937626123 - 1.4208977112157295) / 2.365270937626123 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SystemTextJsonReflection - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/MicroBenchmarks.Serializers.Json_ToString(IndexViewModel).SystemTextJsonReflection.html>) 19.70 μs 17.22 μs 0.87 0.01 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;IndexViewModel&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Json_ToString<IndexViewModel>.SystemTextJson_Reflection_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.224629605640924 < 18.8746436508699. IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 43.20137386213735 (T) = (0 -17153.724904332896) / Math.Sqrt((73711.79671194676 / (299)) + (24038.334900247573 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15185681272707308 = (20225.03412364608 - 17153.724904332896) / 20225.03412364608 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.ReadJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[DeserializeFromReader - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Serialization.Tests.ReadJson(BinaryData).DeserializeFromReader(Mode%3a%20SourceGen).html>) 1.49 μs 1.39 μs 0.93 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;BinaryData&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Serialization.Tests.ReadJson<BinaryData>.DeserializeFromReader(Mode: SourceGen) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.3921005475117165 < 1.4258294424051434. IsChangePoint: Marked as a change because one of 2/8/2023 4:45:44 PM, 3/14/2023 10:43:28 AM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 18.626205673970354 (T) = (0 -1400.7344998886988) / Math.Sqrt((400.195370600494 / (299)) + (126.78035974711689 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.06428687732388312 = (1496.9700284662376 - 1400.7344998886988) / 1496.9700284662376 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Benchstone.BenchF.InvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Test - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Benchstone.BenchF.InvMt.Test.html>) 3.50 ms 3.25 ms 0.93 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.InvMt*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Benchstone.BenchF.InvMt.Test ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.2501926778846153 < 3.32390862344215. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 502.72018080207226 (T) = (0 -3250135.2977764425) / Math.Sqrt((68515783.94305143 / (299)) + (70836.87983507219 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.07136639460009296 = (3499911.352418485 - 3250135.2977764425) / 3499911.352418485 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in MicroBenchmarks.Serializers.Binary_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[BinaryFormatter_ - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/MicroBenchmarks.Serializers.BinaryToStream(Location).BinaryFormatter.html>) 4.97 μs 3.93 μs 0.79 0.17 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;Location&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### MicroBenchmarks.Serializers.Binary_ToStream<Location>.BinaryFormatter_ ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.9261319916695445 < 4.646537441005353. IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 7.0770598860576435 (T) = (0 -4138.864003219656) / Math.Sqrt((22300.602332752453 / (299)) + (50197.878826001004 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14673120419537516 = (4850.598104102406 - 4138.864003219656) / 4850.598104102406 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteBool - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.IO.Tests.BinaryWriterTests.WriteBool.html>) 3.43 ns 0.74 ns 0.21 0.15 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.IO.Tests.BinaryWriterTests.WriteBool ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 0.7374684438178711 < 3.254874801460252. IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 75.11867469352327 (T) = (0 -0.7303053284415864) / Math.Sqrt((0.09051183539199953 / (299)) + (0.004035788892584778 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.7883706236474973 = (3.450869350128148 - 0.7303053284415864) / 3.450869350128148 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[GetHashCodeBenchmark - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Numerics.Tests.Perf_VectorOf(Double).GetHashCodeBenchmark.html>) 12.58 ns 11.08 ns 0.88 0.00 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 11.0750713846162 < 12.176965403431826. IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 40.32829191207483 (T) = (0 -11.004334255324547) / Math.Sqrt((0.019249969269534286 / (299)) + (0.010929769245276437 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14810093059439808 = (12.917415513791598 - 11.004334255324547) / 12.917415513791598 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Logging.Formatting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[NoArguments - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.Logging.Formatting.NoArguments.html>) 43.51 ns 36.22 ns 0.83 0.21 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.Formatting*'
### Payloads [Baseline]() [Compare]() ### Histogram #### Microsoft.Extensions.Logging.Formatting.NoArguments ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 36.21623664183838 < 39.60071820672769. IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 85.02192219068354 (T) = (0 -36.25830579502727) / Math.Sqrt((2.2774540825769014 / (299)) + (0.010682272085040943 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.18803468461008174 = (44.65499339416422 - 36.25830579502727) / 44.65499339416422 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare 1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in V8.Crypto.Support

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Bench - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/V8.Crypto.Support.Bench.html>) 6.01 ms 3.75 ms 0.62 0.00 True

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'V8.Crypto.Support*'
### Payloads [Baseline]() [Compare]() ### Histogram #### V8.Crypto.Support.Bench ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 3.753403972098214 < 5.709020841626017. IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/21/2023 3:58:09 AM falls between 5/12/2023 10:43:40 AM and 5/21/2023 3:58:09 AM. IsImprovementStdDev: Marked as improvement because 424.600253922764 (T) = (0 -3747165.9454159797) / Math.Sqrt((6356756044.132194 / (299)) + (41928470.80253443 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.38155633754188895 = (6059025.53924188 - 3747165.9454159797) / 6059025.53924188 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` ### JIT Disasms ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)