JimBobSquarePants commented 1 week ago

Prerequisites

[x] I have written a descriptive pull-request title
[x] I have verified that there are no overlapping pull-requests open
[x] I have verified that I am following the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules :cop:.
[x] I have provided test coverage for my change (where applicable)

Description

This PR is an attempt to work around issues found with the Mono AOT compiler which causes slow performance on IOS Android

https://github.com/dotnet/runtime/issues/71210

In the linked Issue, Analysis from the Mono team highlighted this indirection as a culprit.

The AOT compiler does appear to have problems figuring out which instances to generate. In this specific case, the caller is ImageDecoderUtilities:Decode<Rgba32> which calls IImageDecoderInternals::Decode on an argument. So in theory, the aot compiler could figure out that the call could possible go to JpegDecoderCore::Decode<Rgba32> and generate that instance. Currently, this kind of analysis is not done.

This PR removes that indirection completely by introducing an internal base class, ImageDecoderCore for all XXDecoderCore instances. In addition, seeding has been introduced for SpectralConverter<TPixel> and others.

This is currently untested but I'm confident that this should improve matters.

beeradmoore commented 6 days ago

I don't think this PR had any impact for what I have tested so far (assuming I built the PR correctly), but I also didn't see the super slow loads the initial person reported. I do still need to test Android.

I have a test repo here. It is a .NET 8 MAUI application. I will test Windows and Mac out of curiosity. I won't test Xamarin.Forms (Xamarin.iOS/Xamarin.Android)

The readme has instructions on how I built this PR into a local nuget package, and how I tested the project on my iPhone 15 Pro Max in both debug and release mode.

The test image I am using is located in ImageSharpMAUITest/ImageSharpMAUITest/Resources/Raw/sloth.jpg

The ImageSharp code I am using is here.

I am loading the image from a stream in the raw folder. I don't think there is any overhead from loading it like this instead of passing file on disk.

using (var stream = await FileSystem.OpenAppPackageFileAsync("sloth.jpg"))
{
    using (var image = await SixLabors.ImageSharp.Image.LoadAsync(stream))
    {
        ...
    }
}

Results

Updating these results as they come in

Debug (this PR)

Device	JpgLoad	JpgResize	PngLoad	PngResize
iPhone	3032.1ms	4060.1ms	26.2ms	73.9ms
iOS Simulator	2884.9ms	3615.2ms	21.7ms	61.5ms
Android	14342.7ms	18562.1ms	123.8ms	348ms
Android Emulator	3193.1ms	3933ms	48.3ms	88.7ms
macOS	2708.2ms	3512.4ms	21.4ms	60.9ms
Windows	106.6ms	82.3ms	26.3ms	17.1ms

Debug (3.1.4)

Device	JpgLoad	JpgResize	PngLoad	PngResize
iPhone	2932.4ms	3895.8ms	25ms	67.7ms
iOS Simulator	2794.2ms	3555.1ms	21.9ms	61.4ms
Android	14289.4ms	18504.5ms	122.7ms	346.4ms
Android Emulator	3027.6ms	4075.2ms	48.9ms	97.6ms
macOS	2701.9ms	3425.3ms	20.6ms	59.3ms
Windows	98.5ms	86.4ms	22.4ms	19.7ms

Release (this pr)

Device	JpgLoad	JpgResize	PngLoad	PngResize
iPhone	62.7ms	80.3ms	4.2ms	4ms
iOS Simulator	64.3ms	81.3ms	3.7ms	4.1ms
Android	1134.5ms	1369.2ms	33.7ms	41.6ms
Android Emulator	243.3ms	300.8ms	19.6ms	25.3ms
macOS	188.5ms	233.9ms	4.9ms	7ms
Windows	149.3ms	96.4ms	16.9ms	14.7ms

Release (3.1.4)

Device	JpgLoad	JpgResize	PngLoad	PngResize
iPhone	61.2ms	78.8ms	4ms	4ms
iOS Simulator	61.1ms	79.6ms	3.4ms	4.1ms
Android	1121.5ms	1349.6ms	35.1ms	43ms
Android Emulator	227.8ms	287.9ms	16.7ms	13.5ms
macOS	181.5ms	230.1ms	5.2ms	7.3ms
Windows	153.1ms	146.4ms	17.2ms	15ms

Test devices

Device	Hardware	OS
iPhone	iPhone 15 Pro Max	17.5.1
iOS Simulator	iPhone 15	17.4
Android	Pixel 2 XL	Android 14
Android Emulator	Pixel 3a	Android 14
macOS	MacBook Pro M3 Max	Sonoma 14.5
Windows	AMD 3300X + RTX 3060	Windows 11 23H2

beeradmoore commented 3 days ago

Added all the numbers.

My takeaway from this (as someone who uses images, but doesn't really know much about image encoding) is:

Jpg is way slower than png
Debug mode in almost all platforms is slower than I would have expected
Jpgs on an android device in debug mode is wild
Shocked to see jpg on android emulator be faster than android on device
I don't see any noticeable difference in the results of this PR and the currently published 3.1.4. I don't know if there is meant to be additional changes to using AOT mode for the tests or something.

More than happy to add and run more tests as requested. Attaching the raw results numbers below (if anyone wants to see the results per run).

ImageSharpMAUITest-Results.zip

JimBobSquarePants commented 14 hours ago

@beeradmoore I've been doing some R&D in order to understand the Android performance. I can't see a configuration value in your sample to enable LLVM which is required for maximum performance. Did I miss something?

https://learn.microsoft.com/en-us/dotnet/android/building-apps/build-properties#enablellvm

beeradmoore commented 13 hours ago

I did not.

I added

    <PropertyGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'android' AND '$(Configuration)' == 'Release'">
        <EnableLLVM>true</EnableLLVM>
    </PropertyGroup>

And during executing all the tests it gets to about 15/40 (half way through jpg load and resize run) and it crashes with

07-16 12:36:03.133  1730  1778 W WindowManager: Exception thrown during dispatchAppVisibility Window{647a9a8 u0 com.beeradmoore.imagesharpmauitest/crc64df7e0c4a761c65fa.MainActivity EXITING}
07-16 12:36:03.133  1730  1778 W WindowManager: android.os.DeadObjectException
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.os.BinderProxy.transactNative(Native Method)
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.os.BinderProxy.transact(BinderProxy.java:586)
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.view.IWindow$Stub$Proxy.dispatchAppVisibility(IWindow.java:552)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.WindowState.sendAppVisibilityToClients(WindowState.java:3217)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.WindowContainer.sendAppVisibilityToClients(WindowContainer.java:1293)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.WindowToken.setClientVisible(WindowToken.java:403)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.ActivityRecord.setClientVisible(ActivityRecord.java:7100)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.ActivityRecord.postApplyAnimation(ActivityRecord.java:5820)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.ActivityRecord.commitVisibility(ActivityRecord.java:5762)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.Transition.finishTransition(Transition.java:1257)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.TransitionController.finishTransition(TransitionController.java:925)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.WindowOrganizerController.finishTransition(WindowOrganizerController.java:489)
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.window.IWindowOrganizerController$Stub.onTransact(IWindowOrganizerController.java:278)
07-16 12:36:03.133  1730  1778 W WindowManager:     at com.android.server.wm.WindowOrganizerController.onTransact(WindowOrganizerController.java:199)
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.os.Binder.execTransactInternal(Binder.java:1500)
07-16 12:36:03.133  1730  1778 W WindowManager:     at android.os.Binder.execTransact(Binder.java:1444)

This is new to me and I have no idea what or why it is doing this. I have re-installed AndroidOS between then and now, this could be related, it could be AOT compile. Tonight I can revert back to not having LLVM and see if to happens again or not. It could also be I did no have developer mode enabled so maybe it was going to sleep in 15sec 🤷‍♂️

I did a single run of with v3.1.4 in release mode running the JpgLoad test and got 1116.8ms. For the local nuget I got 1126.6ms.

I referred to the previous thread and grabbed info from this comment,

and just put

<EnableLLVM>true</EnableLLVM>
<RunAOTCompilation>true</RunAOTCompilation>
<AndroidEnableProfiledAot>false</AndroidEnableProfiledAot>

in my main PropertyGroup to ensure it is enabled.

With that I got 516ms for the local nuget. Huge improvement!

I can test again tonight to see what ones of those above properties are required, if my release Android targeting below was working as intended.

SixLabors / ImageSharp

Mono AOT decoder workaround for slow jpeg decoding. #2762