[Xamarin.Android] Performance decrease when compared with Xamarin.Forms

grendello commented 3 years ago

I recently tested Xamarin.Android NET6 performance to compare it with "legacy" Xamarin.Android. Two test applications use Xamarin.Forms and MAUI - a very simple layout with just a single Label, no resource dictionary. Unfortunately, MAUI appears to be approximately 100ms slower than Xamarin.Forms. Please take a look at the this gist which contains tables comparing performance of different apps (Xamarin.Forms and MAUI)

I plan to do some profiling of the MAUI app next week to find out where the most time is spent.

grendello commented 3 years ago

MAUI test app used in the tests MauiPerfTest.net6.zip

rogihee commented 3 years ago

This sounds exactly like the performance hit taken by using Microsfoft.Extensions.DependencyInjection package. The team are considering using an alternative DI container to circumvent this: #880

AmrAlSayed0 commented 3 years ago

Is it possible for MAUI to depend only on Microsoft.Extensions.DependencyInjection.Abstractions and provide another package for the concrete implementation of Microsoft.Extensions.DependencyInjection so that other DI packages can just implement the abstraction and configure it to be used with MAUI?

This could be worth the effort if that's actually what's causing the performance issue. Microsoft.Extensions.DependencyInjection runs really quickly on Linux and Windows on .NET Core but it is not as quick on Android and iOS.

grendello commented 3 years ago

I'm not a big fan of DI, especially on mobile, but I realize it's a core design paradigm in XF/MAUI so it's here to stay. However, runtime detection of renderers (and other providers etc) in mobile apps makes for a large time and resource loss, because a mobile app is effectively frozen in time when it's installed. The app won't change unless it's reinstalled (for the most part - unless it downloads some assemblies/native libraries from the internet when it runs, which I hope no apps do) and so everything about it is known statically. Would it be possible to take advantage of .NET6 code generators on the build time to output code that tells MAUI everything it needs to know, so that it doesn't have to dynamically look on startup?

rogihee commented 3 years ago

They have looked into both options (other package and generators), but unfortunately the Microsoft.Extensions.DependencyInjection.Abstractions is not well suited for source generators in its current form. Switching means a breaking change and makes the eco-system weaker bc integrating Blazor will get harder.

For a good discussion and performance metrics see https://github.com/dotnet/runtime/issues/44432.

My gut feeling is that they are trying to squeeze more performance out of the current design. David Fowler was speaking of 10x improvement for reflection, that must have an effect here...

grendello commented 3 years ago

@AmrAlSayed0 quite some time ago I profiled Xamarin.Forms and the bulk of startup time was spent looking for types in assemblies that need to be registered. I haven't been able to profile MAUI yet (since .NET6 doesn't have a managed profiler for mobile that ships with it) but I will hopefully be able to do it soon. Dynamic DI involves a lot of I/O which is generally VERY slow on mobile (especially with the mid to low tier phones, but even high-end devices are blazingly very fast). Comparing Linux/Windows performance to mobile devices, thus, is not a good measure. Keep in mind that mobile devices want to first and foremost preserve the battery, so ideally they spend most time in as low power mode as possible. That means stepping down (and often turning off) CPU cores which are hot-plugged and stepped up only when needed (that's why mobile devices use the BIG.little architecture - they have a number of "slow" but low energy cores and a smaller number of high performance cores). This also includes power delivered to storage, WiFi, RAM, display etc. So when an application suddenly puts a demand for a large amount of I/O, the system needs to wake everything up and face the demand. If the application heavily uses threads and puts a real load on each thread, the system will have to wake up the CPU cores and crank their clocks up to meed the demand. All of this takes time and slows down, especially, application startup.

Imagine a MAUI app that provides a notification service to the user. The app isn't running in the background, in fact, it might not be actually running most of the time - it wakes up only when some notification (or intent, on Android) is delivered and the application has to start in background. The simple MAUI app above takes around 1s to start, one with 5 more controls takes 1.5s. The startup performance is absolutely crucial also because it's the first impression of the end user about the app.

It's crucial for a mobile app to

start as quickly as possible
do as little work as possible
do absolute minimum of necessary work especially on startup
do not use threads to make things "faster" - use threads to offload long running/blocking operations, but avoid threads on startup as much as possible

grendello commented 3 years ago

@rogihee MAUI is where breaking changes can, and should, happen but before it is released. NET6 is an opportunity to introduce breaking changes (and they are introduced) wherever it makes sense, like here. Also, source generators aren't the only option. It should be possible to introduce a post-build step (in the likeness of the managed linker) that runs over all the gathered assemblies and generates another assembly with the necessary information in as efficient form as possible (i.e. no LINQ, no parsing of strings, no reading of metadata, no reflection, no I/O etc). That single assembly is loaded at startup and its data accessed lazily. The data could be a serialized dictionary (binary serialization, no XML) or a simple array (less efficient, of course), or any other form of a data structure that's efficient to load (preferably in bulk) into memory and accessed quickly in a lazy manner. The generator could analyze the code that takes part in the startup (e.g. the main/launcher activity on Android), look which types are required for startup and prioritize that data (possibly putting it in a smaller, separate, data structure) so that the startup is faster. Even with more complexity involved in MAUI, there's no reason it has to be 5 times slower on startup than "pure" Xamarin.Android (which currently starts in around 300ms for a comparable application with one control and only slightly slower for a more complex app with a few controls)

rogihee commented 3 years ago

@grendello agree 100%, I have been chatting with the team in Discord and making this very point.

grendello commented 3 years ago

I will hopefully be able to profile some MAUI samples soon and then we can see where it gets the hit

grendello commented 3 years ago

Another idea for post-processing, a more complex one, is patching the application code after the build (e.g. with Cecil) so that references to the DI data are injected directly into the code (effectively replacing custom attributes). For instance (and I'm inventing it as I go because I don't know the specifics of MAUI code), whenever a renderer is needed instead of kicking off the DI registration run (e.g. with a call to Registerar.FindType (Some.Type.Here)), the call is replaced with a direct type instantiation.

I remember that Xamarin.Forms code had quite a few instances of dynamic platform checks, too (IsIOS or IsAndroid calls IIRC). These can could be eliminated in a post-processing step too (admittedly, this is trickier than the above FindType suggestion) as they are a waste of CPU cycles in a mobile app - the app, as it is, will not migrate from Android to iOS and vice versa.

akoeplinger commented 3 years ago

I remember that Xamarin.Forms code had quite a few instances of dynamic platform checks, too (IsIOS or IsAndroid calls IIRC). These can could be eliminated in a post-processing step too (admittedly, this is trickier than the above FindType suggestion) as they are a waste of CPU cycles in a mobile app - the app, as it is, will not migrate from Android to iOS and vice versa.

If you use the new .NET5+ OperatingSystem.IsAndroid() and .IsIOS() APIs then these are simple hardcoded bools now (and the managed linker should be able to get rid of unused codepaths)

grendello commented 3 years ago

I've managed to get the new NET6 EventPipe profiler working with Xamarin.Android and profile the HelloMaui sample on a Pixel 3 XL device. The data is for a 64-bit Release build of the app. According to Android's displayed time, the app took 1.6s to start:

06-17 10:08:47.209  1750  1789 I ActivityTaskManager: Displayed com.microsoft.hellomaui/crc64ec6bb0d0f3cda3b3.MainActivity: +1s603ms

However, the above startup time is for the app when built with the Xamarin.Android property AndroidLinkResources set to true (it's false by default). With the property set to false, startup time reaches above 2s. The app was built using the EventPipe branch, with dotnet 6.0.100-preview.6.21313.2 using the following command line:

dotnet build -f net6.0-android \
        /t:Install \
        /bl \
        /p:Configuration=Release \
        /p:Enable64BitBuild=true \
        /p:AndroidLinkResources=true \
        /p:AndroidEnableProfiler=true

In attachment you can find a zip with both the .nettrace and speedscope formatted data. The former can be loaded into Perfview on Windows, the latter can be opened in https://speedscope.app

I haven't looked at MAUI code yet, but from looking at the "Sandwich" tab in https://speedscope.app, with data sorted in descending order on Total, the thing that stands out is a very high reflection usage, the following calls:

System.Private.CoreLib!System.Reflection.RuntimeConstructorInfo.InternalInvoke(object,object[],bool)
System.Private.CoreLib!System.Activator.CreateInstance(System.Type)

All of the calls appear to be rooted in microsoft.maui!Microsoft.Maui.MauiAppCompatActivity.OnCreate(Android.OS.Bundle) which is more or less the same what was happening in Xamarin.Forms. If you click on the entries, you can see the call graph on the right-hand side of the screen (in Speedscope), with the entry call at the top, inner calls descending toward the bottom of the screen. By looking at the call graphs it can be seen that a lot of reflection comes from dependency injection as well as from XAML loaders.

If there was a way of eliminating all of reflection use (and I imagine it's perfectly possible using code generated at build time) I think it should be realistic to achieve around 500ms startup time for MAUI (with plain Xamarin.Android startup time on Pixel 3 XL being around 300ms). What worried me when I looked at MAUI startup time previously, was that the time appeared to be a function of the number of controls used in the start activity. On one hand it's logical - there needs to be more reflection and and instantiations, but on the other hand this is not something that's viable on the long run. Application startup should not be affected by the number of managed types instantiate on its initial screen (it can, of course, depend on what the controls do - but that's beyond the scope of both MAUI and Xamarin.Android teams work), at least not to such degree as I observed before (a MAUI app with a single control took around 900ms to start while Hello Maui was at 1.5s by adding a few controls). I believe that replacing reflection with generated code that works lazily would make the startup time much, much better.

hellomaui-profile.zip

lambdageek commented 3 years ago

with data sorted in descending order on Total, the thing that stands out is a very high reflection usage,

For Invoke and CreateInstance calls, wouldn't the total time include the time spent inside the constructors themselves?

There are a couple of low-hanging fruit that stood out in the stack trace:

Should this ToLower be ToLowerInvariant()? (maybe not - I'm not sure if "title" here is localizable)

https://github.com/dotnet/maui/blob/18e0f4ebbcca7c904ccdb67e6439cb72382e2a40/src/Compatibility/Core/src/Android/ResourceManager.cs#L383
Font loading (Microsoft.Maui.EmbeddedFontLoader..ctor) calls System.IO.Path.GetTempPath which calls the static constructor for System.IO.PathInternal which then writes a file (!) to find out whether the filesystem is case-sensitive. Can we just always return true on Android?

https://github.com/dotnet/runtime/blob/386d0b19205c0ec95617eb25e94dca32ce823175/src/libraries/Common/src/System/IO/PathInternal.CaseSensitivity.cs#L40

grendello commented 3 years ago

with data sorted in descending order on Total, the thing that stands out is a very high reflection usage,

For Invoke and CreateInstance calls, wouldn't the total time include the time spent inside the constructors themselves?

Yes, it's very likely, but it can also include "backend" processing in e.g. XA runtime (type registration)

There are a couple of low-hanging fruit that stood out in the stack trace:
1. Should this `ToLower` be `ToLowerInvariant()`? (maybe not - I'm not sure if "title" here is localizable)
Yep, that should make a nice difference.

https://github.com/dotnet/maui/blob/18e0f4ebbcca7c904ccdb67e6439cb72382e2a40/src/Compatibility/Core/src/Android/ResourceManager.cs#L383
1. Font loading (`Microsoft.Maui.EmbeddedFontLoader..ctor`) calls `System.IO.Path.GetTempPath` which calls the static constructor for `System.IO.PathInternal` which then writes a file (!) to find out whether the filesystem is case-sensitive.  Can we just always return `true` on Android?

Yes and no. Internal filesystems are ext4 on Android, case-sensitive by default (ext4 supports case-insensitivity) but other filesystems may be e.g. exFAT or f2fs which are case-insensitive. Normally you could use /sys/fs/[FSTYPE]/features to see if case folding is supported, but Android blocks access to it for "security". That said, I think it might be better to look at what fs is mounted at the specific path? Storage I/O should be avoided as much as possible.

https://github.com/dotnet/runtime/blob/386d0b19205c0ec95617eb25e94dca32ce823175/src/libraries/Common/src/System/IO/PathInternal.CaseSensitivity.cs#L40

taublast commented 2 years ago

The problem might also come from the new compiler, or android related libs and tools. All my released forms android apps used to have a startup of around 4 secs with custom profile aot or full aot, didn't matter. Theses are huge with DI and many custom renderers and assemblies, so it's still fine. As of today same apps, pulled from git and built in VS 2022 start time became 9-15 secs, impossible to publish. Changing the following is not affecting this startup time:

version of Microsoft.Extensions.Hosting and related nugets 3.x-6.x for DI
xamarin forms 4.8.0.1687 - latest
using custom aot profile from git commit or generating new with actual tools

Now im still at xamarin, not maui but who knows maybe it's the same cause.

UPDATE This was due to Visual Studio 22 Preview 17.2. After installing and recompiling in Visual Studio 22 17.1 everything went back to normal, app with DI without much lib assemblies starts in 2 secs! Looking in fear to the upcoming vs stable update to 17.2...

jonathanpeppers commented 2 years ago

@taublast could you file a new issue with the problem you're facing here: https://github.com/xamarin/xamarin-android/issues

This issue is more about tracking .NET MAUI performance. We're actively working on that.

jonathanpeppers commented 2 years ago

For updates on .NET MAUI performance, I have lots of numbers here:

https://github.com/jonathanpeppers/maui-profiling

The next preview TBA.

jonathanpeppers commented 2 years ago

MAUI is faster than Xamarin.Forms (especially in RC1 or newer), going to close this, thanks everyone!

dotnet / maui

[Xamarin.Android] Performance decrease when compared with Xamarin.Forms #822