search

dotnet / perf-autofiling-issues

A landing place for auto-filed performance issues before they receive triage
MIT License
9 stars 4 forks source link

[Perf] Linux/arm64: 6 Improvements on 10/6/2022 5:26:59 PM #8900

Closed performanceautofiler[bot] closed 2 years ago

performanceautofiler[bot] commented 2 years ago

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 8eb413818f5b95e750be5cf4148a3a9714ddc331
Compare e43178aa3e521bb7fe9b9cdc2474e626178520b9
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetService

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.DependencyInjection.GetService.Scoped.html>) 95.92 ns 74.38 ns 0.78 0.32 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetService*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Microsoft.Extensions.DependencyInjection.GetService.Scoped ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 74.37588176372321 < 92.46374172776798. IsChangePoint: Marked as a change because one of 9/13/2022 2:36:07 PM, 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 20.884675478620903 (T) = (0 -75.80505294053816) / Math.Sqrt((75.25034256480811 / (299)) + (4.418949486222967 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.2269473826810935 = (98.05937040022515 - 75.80505294053816) / 98.05937040022515 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 8eb413818f5b95e750be5cf4148a3a9714ddc331
Compare e43178aa3e521bb7fe9b9cdc2474e626178520b9
Diff Diff

Improvements in PerfLabTests.EnumPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[ObjectGetType - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/PerfLabTests.EnumPerf.ObjectGetType.html>) 2.67 ns 0.33 ns 0.13 0.53 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.EnumPerf*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### PerfLabTests.EnumPerf.ObjectGetType ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 0.33401263324163794 < 2.5674050152804293. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 29.50958448390081 (T) = (0 -0.13368588824962643) / Math.Sqrt((0.705683394891501 / (299)) + (0.033360122328029665 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.9545009034203477 = (2.938209729408389 - 0.13368588824962643) / 2.938209729408389 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 8eb413818f5b95e750be5cf4148a3a9714ddc331
Compare e43178aa3e521bb7fe9b9cdc2474e626178520b9
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Factory - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory.html>) 56.48 ns 47.13 ns 0.83 0.31 False
[ServiceProvider - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider.html>) 41.10 ns 35.78 ns 0.87 0.32 False

graph graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.Factory ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 47.13273031127675 < 53.63531881686322. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 8.179803256583305 (T) = (0 -49.373225849747236) / Math.Sqrt((11.972748361508938 / (299)) + (4.932232824804725 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.14372769050820186 = (57.66065923473631 - 49.373225849747236) / 57.66065923473631 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ```#### Microsoft.Extensions.DependencyInjection.ActivatorUtilitiesBenchmark.ServiceProvider ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 35.78124900809317 < 38.86652791689329. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 9:08:45 AM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 6.601376816537162 (T) = (0 -36.65553583328826) / Math.Sqrt((2.8785452462104866 / (299)) + (1.8028885188970443 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.09114760382059654 = (40.33167100332167 - 36.65553583328826) / 40.33167100332167 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 8eb413818f5b95e750be5cf4148a3a9714ddc331
Compare e43178aa3e521bb7fe9b9cdc2474e626178520b9
Diff Diff

Improvements in Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[Scoped - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped.html>) 90.05 ns 83.20 ns 0.92 0.39 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### Microsoft.Extensions.DependencyInjection.GetServiceIEnumerable.Scoped ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 83.20136786121003 < 85.58257972143467. IsChangePoint: Marked as a change because one of 9/22/2022 3:11:11 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 4.581511916335929 (T) = (0 -76.62908476010452) / Math.Sqrt((30.848851461004024 / (299)) + (35.23945837452018 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.13784359921694492 = (88.88072360247632 - 76.62908476010452) / 88.88072360247632 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 8eb413818f5b95e750be5cf4148a3a9714ddc331
Compare e43178aa3e521bb7fe9b9cdc2474e626178520b9
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
[IsDefinedClassMiss - Duration of single invocation](<https://pvscmdupload.z22.web.core.windows.net/reports/allTestHistory/refs/heads/main_arm64_ubuntu 20.04/System.Reflection.Attributes.IsDefinedClassMiss.html>) 152.75 ns 138.57 ns 0.91 0.14 False

graph Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Reflection.Attributes*'

Related Issues

Regressions

### Payloads [Baseline]() [Compare]() ### Histogram ### Edge Detector Info [Collection Data]() #### System.Reflection.Attributes.IsDefinedClassMiss ```log ``` ### Description of detection logic ```IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small. IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small. IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline. IsImprovementWindowed:Marked as improvement because 138.5742613842758 < 145.24239307225506. IsChangePoint: Marked as a change because one of 8/5/2022 12:58:40 PM, 9/22/2022 11:37:30 PM, 9/27/2022 9:53:04 AM, 10/6/2022 2:13:23 PM, 10/13/2022 2:25:23 AM falls between 10/4/2022 9:16:29 AM and 10/13/2022 2:25:23 AM. IsImprovementStdDev: Marked as improvement because 29.58604553132004 (T) = (0 -137.90711371861934) / Math.Sqrt((35.44474555729882 / (299)) + (0.5804674907076517 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.09413727613349147 = (152.2384243056036 - 137.90711371861934) / 152.2384243056036 is greater than 0.05. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so. ``` ### Docs [Profiling workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/profiling-workflow-dotnet-runtime.md) [Benchmarking workflow for dotnet/runtime repository](https://github.com/dotnet/performance/blob/master/docs/benchmarking-workflow-dotnet-runtime.md)
EgorBo commented 2 years ago

https://github.com/dotnet/runtime/pull/76649