search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/x64: 4 Improvements on 9/29/2022 2:21:18 AM #8866

Open performanceautofiler[bot] opened 1 year ago

performanceautofiler[bot] commented 1 year ago

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 91a07a71153ae28f599c2540b238755a207d5c0b
Compare da9b348cbd164aa9fed9242c52d8a101b1f00561
Diff Diff

Improvements in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Char_ToUpper - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToUpper(c%3a%20%27A%27%2c%20cultureName%3a%20en-US).html>) 155.94 ns 138.92 ns 0.89 0.25 False
[GetUnicodeCategory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.GetUnicodeCategory(c%3a%20%27.%27).html>) 65.46 ns 60.03 ns 0.92 0.08 False
[Char_ToLowerInvariant - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.Tests.Perf_Char.Char_ToLowerInvariant(input%3a%20%22Hello%20World!%22).html>) 902.84 ns 801.52 ns 0.89 0.10 False

graph graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Char*'
### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Tests.Perf_Char.Char_ToUpper(c: 'A', cultureName: en-US) ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 138.91969208547755 < 143.8541507301458. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 5.479273678234253 (T) = (0 -136.3339468129565) / Math.Sqrt((161.17345564461294 / (299)) + (111.0431345628216 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08517694281546924 = (149.02766796514652 - 136.3339468129565) / 149.02766796514652 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.GetUnicodeCategory(c: '.') ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 60.02681065478585 < 62.37840149388259. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 20.867665473943013 (T) = (0 -59.827180500551464) / Math.Sqrt((5.367180587371816 / (299)) + (1.8781321931733983 / (24))) is greater than 1.967381707010986 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (24) - 2, .975) and 0.09762546497539834 = (66.29972165484521 - 59.827180500551464) / 66.29972165484521 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!") ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 801.5242498267037 < 855.9842445597626. IsChangePoint: Marked as a change because one of 9/28/2022 6:40:13 PM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 29.763342905642013 (T) = (0 -814.3839144479035) / Math.Sqrt((852.8366540684299 / (299)) + (192.69007252512702 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.10481254366307344 = (909.7356187053066 - 814.3839144479035) / 909.7356187053066 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 91a07a71153ae28f599c2540b238755a207d5c0b
Compare da9b348cbd164aa9fed9242c52d8a101b1f00561
Diff Diff

Improvements in System.IO.Tests.Perf_File

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ReadAllLines - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_x64_ubuntu 18.04_CompliationMode=wasm_RunKind=micro/System.IO.Tests.Perf_File.ReadAllLines.html>) 219.83 μs 202.47 μs 0.92 0.04 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_File*'
### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.IO.Tests.Perf_File.ReadAllLines ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 202.47096600274728 < 208.82448133352193. IsChangePoint: Marked as a change because one of 9/28/2022 9:30:44 PM, 10/3/2022 1:11:02 PM falls between 9/24/2022 8:58:25 PM and 10/3/2022 1:11:02 PM. IsImprovementStdDev: Marked as improvement because 42.747062232801035 (T) = (0 -203846.0447142184) / Math.Sqrt((6752516.082607396 / (299)) + (2883894.6516651157 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.07464484840536581 = (220289.52274479394 - 203846.0447142184) / 220289.52274479394 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
lewing commented 1 year ago

some small wins from https://github.com/dotnet/runtime/pull/76274