search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 25 Improvements on 4/17/2023 5:51:51 AM #16786

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20False%2c%20SkipValidation%3a%20False).html>) 25.50 ms 23.15 ms 0.91 0.00 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20False).html>) 26.19 ms 23.80 ms 0.91 0.00 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20True%2c%20SkipValidation%3a%20True).html>) 26.02 ms 23.53 ms 0.90 0.00 False
[WriteDoubles - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted%3a%20False%2c%20SkipValidation%3a%20True).html>) 25.41 ms 23.00 ms 0.91 0.00 False

graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.14960621153846 < 24.406127528241758. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 115.83046550000645 (T) = (0 -23195707.532941595) / Math.Sqrt((228137413417.20987 / (299)) + (3000422742.4469547 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.14188642221712383 = (27031045.92852705 - 23195707.532941595) / 27031045.92852705 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.799010483333333 < 24.90676278082156. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 127.7220482999024 (T) = (0 -23819831.703439157) / Math.Sqrt((226203778051.22702 / (299)) + (1220641312.7461925 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.13804849633965152 = (27634770.172435768 - 23819831.703439157) / 27634770.172435768 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.533973785714288 < 24.764580053958333. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 122.5397136884014 (T) = (0 -23425429.8468254) / Math.Sqrt((229585010352.88095 / (299)) + (3008389618.2501516 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.14796496428933853 = (27493505.390053384 - 23425429.8468254) / 27493505.390053384 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 23.003832875 < 24.181085267121595. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/6/2023 1:55:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 99.66578540894378 (T) = (0 -22912160.469820924) / Math.Sqrt((228928432769.73398 / (299)) + (7369055386.382166 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1475924047563754 = (26879348.093176544 - 22912160.469820924) / 26879348.093176544 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringHebrewIsrael - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael.html>) 387.37 ns 348.34 ns 0.90 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20).html>) 241.26 ns 187.31 ns 0.78 0.01 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20da).html>) 207.08 ns 171.36 ns 0.83 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20fr).html>) 240.65 ns 187.72 ns 0.78 0.02 False
[ToString - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring%3a%20ja).html>) 236.01 ns 185.45 ns 0.79 0.02 False

graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 348.3403418737118 < 360.9854493207077. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 3/23/2023 3:52:33 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 34.36422379782819 (T) = (0 -355.22161960428485) / Math.Sqrt((64.16609961095082 / (299)) + (8.301120473144085 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.09350660634464404 = (391.8634400322373 - 355.22161960428485) / 391.8634400322373 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 187.31175904413502 < 231.41248615113523. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 2/17/2023 12:28:21 PM, 3/21/2023 2:53:21 PM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 31.87954130077206 (T) = (0 -194.39202815408137) / Math.Sqrt((27.05742577850269 / (299)) + (7.200764937884644 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1340220994901093 = (224.47689258539125 - 194.39202815408137) / 224.47689258539125 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: da) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 171.35554196478182 < 199.91061191588457. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 2/17/2023 12:28:21 PM, 3/21/2023 2:53:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 19.857573199824188 (T) = (0 -181.94959543657268) / Math.Sqrt((15.03653412837575 / (299)) + (18.5480088422483 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1438155202801052 = (212.5121393185011 - 181.94959543657268) / 212.5121393185011 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 187.72088516791797 < 230.54084475082382. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 2/17/2023 12:28:21 PM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 31.31846241604119 (T) = (0 -194.86108108328702) / Math.Sqrt((26.519038277335998 / (299)) + (7.401115744105739 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.13300296769108325 = (224.75403469876784 - 194.86108108328702) / 224.75403469876784 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: ja) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 185.4505863952477 < 229.06500036657974. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 2/17/2023 12:28:21 PM, 3/21/2023 2:53:21 PM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 22.474490019710817 (T) = (0 -194.04787529179976) / Math.Sqrt((32.886168196525304 / (299)) + (10.5451673747578 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.11592063043425571 = (219.49146419638225 - 194.04787529179976) / 219.49146419638225 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.TryGetValueFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 33.01 μs 30.42 μs 0.92 0.54 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 30.418866801000558 < 35.11826751173708. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 25.976537939628532 (T) = (0 -30056.933853517014) / Math.Sqrt((16783819.165338527 / (299)) + (74214.31969878491 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.17984737201506948 = (36647.97603266264 - 30056.933853517014) / 36647.97603266264 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%201.7976931348623157E%2b308).html>) 239.17 ns 212.61 ns 0.89 0.00 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%2012345).html>) 142.24 ns 124.22 ns 0.87 0.00 True
[FormatterUInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value%3a%2018446744073709551615).html>) 107.98 ns 34.41 ns 0.32 0.01 False
[FormatterDouble - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value%3a%20-1.7976931348623157E%2b308).html>) 244.17 ns 213.79 ns 0.88 0.00 False
[FormatterInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value%3a%202147483647).html>) 23.97 ns 17.11 ns 0.71 0.03 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%209223372036854775807).html>) 54.14 ns 40.41 ns 0.75 0.01 False
[FormatterInt64 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value%3a%20-9223372036854775808).html>) 104.12 ns 41.80 ns 0.40 0.06 False
[FormatterUInt32 - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value%3a%204294967295).html>) 52.80 ns 14.50 ns 0.27 0.02 False

graph graph graph graph graph graph graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 212.6125214394205 < 225.97990892169096. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 116.44574544064142 (T) = (0 -212.5506019367768) / Math.Sqrt((13.07763252371781 / (299)) + (0.3235217344579663 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.13715010468966246 = (246.33554815502367 - 212.5506019367768) / 246.33554815502367 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 124.22171556968108 < 135.01429923098578. IsChangePoint: Marked as a change because one of 1/17/2023 4:06:07 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 88.27348423302574 (T) = (0 -122.73407385392255) / Math.Sqrt((0.8261020721758552 / (299)) + (0.44701599352573296 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.14140056418585661 = (142.94683729618728 - 122.73407385392255) / 142.94683729618728 is greater than 0.05. IsChangeEdgeDetector: Marked as regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt64(value: 18446744073709551615) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 34.41474464493457 < 102.47673123980832. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 22.459350884391938 (T) = (0 -34.356421141259496) / Math.Sqrt((150.81391677141534 / (299)) + (0.03497724488754161 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.3178993145102396 = (50.368548034211365 - 34.356421141259496) / 50.368548034211365 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 213.78640363216905 < 230.1516471418295. IsChangePoint: Marked as a change because one of 2/17/2023 12:28:21 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 68.99837905306335 (T) = (0 -213.5755429923579) / Math.Sqrt((12.289248076197653 / (299)) + (1.9671230358456298 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1413552947787354 = (248.73564315210155 - 213.5755429923579) / 248.73564315210155 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: 2147483647) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 17.112204085819005 < 22.562519937678505. IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/21/2023 1:51:00 AM, 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 64.8981348629798 (T) = (0 -16.631480593634617) / Math.Sqrt((3.3399524668090885 / (299)) + (0.23115269494448706 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.42827969474987493 = (29.0902394770786 - 16.631480593634617) / 29.0902394770786 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: 9223372036854775807) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 40.40617060960405 < 51.63139730543768. IsChangePoint: Marked as a change because one of 1/15/2023 4:47:52 PM, 1/21/2023 1:51:00 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 92.61051660824879 (T) = (0 -40.00801496310261) / Math.Sqrt((2.7121419947944916 / (299)) + (0.18129884662935067 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.2919617853705196 = (56.50544580286388 - 40.00801496310261) / 56.50544580286388 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt64(value: -9223372036854775808) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 41.79990859280058 < 99.08511483194663. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 14.617400916814988 (T) = (0 -41.23741484165702) / Math.Sqrt((130.0202039858525 / (299)) + (0.08188284522266373 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.1912547430642821 = (50.98937457500876 - 41.23741484165702) / 50.98937457500876 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterUInt32(value: 4294967295) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 14.501426811702828 < 50.508667188451504. IsChangePoint: Marked as a change because one of 4/9/2023 1:38:49 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 15.98236310333611 (T) = (0 -14.20193978391622) / Math.Sqrt((48.1345770746976 / (299)) + (0.053239674865428334 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.314951539330754 = (20.73129216295426 - 14.20193978391622) / 20.73129216295426 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[StringFormat - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Enum.StringFormat(value%3a%2032).html>) 321.27 ns 285.33 ns 0.89 0.27 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Enum.StringFormat(value: 32) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 285.32664500714003 < 291.9611680680266. IsChangePoint: Marked as a change because one of 3/2/2023 11:15:04 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 35.07674903794472 (T) = (0 -286.73114195033367) / Math.Sqrt((88.24384444360955 / (299)) + (1.3208644715406563 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.07663205783651707 = (310.52750356321957 - 286.73114195033367) / 310.52750356321957 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ToStringHex - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Int32.ToStringHex(value%3a%202147483647).html>) 30.54 ns 26.94 ns 0.88 0.13 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Int32.ToStringHex(value: 2147483647) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 26.939858619971446 < 28.916049452603016. IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 3/16/2023 8:14:39 PM, 3/17/2023 1:30:52 PM, 4/8/2023 3:10:58 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 7.778406242221789 (T) = (0 -27.053776801634143) / Math.Sqrt((1.0619415472504152 / (299)) + (0.28459360176800236 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.053811165432136036 = (28.59236530093904 - 27.053776801634143) / 28.59236530093904 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ImmutableSortedDictionary - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Collections.ContainsKeyFalse(Int32%2c%20Int32).ImmutableSortedDictionary(Size%3a%20512).html>) 40.43 μs 29.84 μs 0.74 0.54 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 29.836072282953104 < 38.331402157519. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 27.83307105933311 (T) = (0 -29741.412562302332) / Math.Sqrt((18871754.33261129 / (299)) + (3511.7894155387075 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.19088982327250534 = (36758.17387762153 - 29741.412562302332) / 36758.17387762153 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
performanceautofiler[bot] commented 1 year ago

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Skip_One - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Linq.Tests.Perf_Enumerable.Skip_One(input%3a%20IEnumerable).html>) 1.19 μs 1.13 μs 0.94 0.61 False
[Min - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Linq.Tests.Perf_Enumerable.Min(input%3a%20IEnumerable).html>) 1.71 μs 658.01 ns 0.38 0.69 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 1.1254651029645122 < 1.485857469331583. IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 30.57792658593479 (T) = (0 -1116.5962395196618) / Math.Sqrt((50270.3670717579 / (299)) + (177.71133272208556 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.27291637881127817 = (1535.7191483616684 - 1116.5962395196618) / 1535.7191483616684 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` #### System.Linq.Tests.Perf_Enumerable.Min(input: IEnumerable) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 658.0109881549274 < 1.3010582506225334. IsChangePoint: Marked as a change because one of 2/18/2023 8:58:19 AM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 22.236893771250454 (T) = (0 -672.9109933132424) / Math.Sqrt((112568.40076660663 / (299)) + (350.13408421020023 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.4024528393725323 = (1126.1219827513485 - 672.9109933132424) / 1126.1219827513485 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[SequenceEqual - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Memory.Span(Int32).SequenceEqual(Size%3a%2033).html>) 8.99 ns 7.02 ns 0.78 0.37 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Memory.Span<Int32>.SequenceEqual(Size: 33) ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 7.024112813085339 < 8.599578024514447. IsChangePoint: Marked as a change because one of 3/3/2023 4:19:26 AM, 4/6/2023 7:14:22 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 6.471476406085769 (T) = (0 -6.971972317579741) / Math.Sqrt((0.3856931128618974 / (299)) + (0.02583190198014001 / (8))) is greater than 1.967772354556772 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (8) - 2, .975) and 0.05873252066701613 = (7.407004353874345 - 6.971972317579741) / 7.407004353874345 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 0b081118dedcbc01d445f3f56222fde9d86f3f44
Compare e3f55cbfd83162bc85d73d69c765e7f44c478e6d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Reverse - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Tests.Perf_Array.Reverse.html>) 51.75 ns 33.24 ns 0.64 0.06 False

graph Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'
### Payloads [Baseline]() [Compare]() ### Histogram #### System.Tests.Perf_Array.Reverse ```log ``` ### Description of detection logic ``` IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed: Marked as improvement because 33.236945049756045 < 49.269197043704125. IsChangePoint: Marked as a change because one of 1/5/2023 10:03:03 PM, 1/21/2023 9:40:41 PM, 1/27/2023 7:12:54 PM, 2/7/2023 2:48:42 AM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/20/2023 12:14:19 AM falls between 4/11/2023 5:52:15 AM and 4/20/2023 12:14:19 AM. IsImprovementStdDev: Marked as improvement because 9.758710024342745 (T) = (0 -33.244743242058526) / Math.Sqrt((72.14961279952067 / (299)) + (0.09874507552766083 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.1284924106075633 = (38.14624639727439 - 33.244743242058526) / 38.14624639727439 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)