See if we can use `$(AndroidAotMode)=full` `MONO_AOT_MODE_FULL`

jonathanpeppers commented 3 weeks ago

Android framework version

net9.0-android

Affected platform version

.NET 9 RC 2

Description

I was trying to enable "full AOT" and disable JIT on Android:

dotnet new maui - remove the other platforms beside Android
dotnet build build -c Release -p:AndroidAotMode=full -p:AndroidEnableProfiledAot=false -p:RunAOTCompilation=true -r android-arm64 -bl -t:Run

This should "AOT everything" and also toggle MONO_AOT_MODE_FULL at runtime.

This appears to pass the right flags to the <MonoAOTCompiler/> task:

And at runtime:

10-30 08:58:14.982  9378  9378 D monodroid: Mono AOT mode: full
...
10-30 08:58:14.982  9378  9378 D monodroid: Probing for Mono AOT mode
10-30 08:58:14.982  9378  9378 D monodroid: Enabling AOT mode in Mono
10-30 08:58:14.982  9378  9378 D monodroid: Probing if we should use LLVM

https://github.com/dotnet/android/blob/aa668a5c746a7669a145e63efe59d0964093e604/src/native/monodroid/monodroid-glue.cc#L1479

But then the JIT is still used and we crash:

10-30 08:58:15.015  9378  9378 F mono-rt : [ERROR] FATAL UNHANDLED EXCEPTION: System.ExecutionEngineException: Attempting to JIT compile method '(wrapper other) void Java.Interop.JavaVMInterface:PtrToStructure (intptr,object)' while running in aot-only mode. See https://learn.microsoft.com/xamarin/ios/internals/limitations for more information.
10-30 08:58:15.015  9378  9378 F mono-rt : 
10-30 08:58:15.015  9378  9378 F mono-rt :    at System.Runtime.InteropServices.Marshal.PtrToStructure[JavaVMInterface](IntPtr )
10-30 08:58:15.015  9378  9378 F mono-rt :    at Java.Interop.JniRuntime.CreateInvoker(IntPtr )
10-30 08:58:15.015  9378  9378 F mono-rt :    at Java.Interop.JniRuntime..ctor(CreationOptions )
10-30 08:58:15.015  9378  9378 F mono-rt :    at Android.Runtime.AndroidRuntime..ctor(IntPtr , IntPtr , IntPtr , IntPtr , Boolean )
10-30 08:58:15.015  9378  9378 F mono-rt :    at Android.Runtime.JNIEnvInit.Initialize(JnienvInitializeArgs* )

Perhaps it fails on the first generic method?

https://github.com/dotnet/java-interop/blob/2bdf2bc6932e29d0666db2761e57b163ba9d76fa/src/Java.Interop/Java.Interop/JniRuntime.cs#L272

Steps to Reproduce

See above.

Did you find any workaround?

No.

Relevant log output

aot-mode-full.txt

jonathanpeppers commented 3 weeks ago

If we got this working, it would probably solve:

https://github.com/dotnet/runtime/issues/101135

jonathanpeppers commented 2 weeks ago

After some conversation, it may be more appropriate for Android to use $(AndroidAotMode)=hybrid anyway. We don't really want people to have to opt into the interpreter, if the JIT can be available.

Some conversation here:

https://discord.com/channels/732297728826277939/732297837953679412/1301214830698696829

Of course, the drawback to using hybrid is app size!

https://discord.com/channels/732297728826277939/732297837953679412/1301294311521910814

Summary:
  +           0 Other entries 0.00% (of 2,858,353)
  +           0 Dalvik executables 0.00% (of 15,406,204)
  +  43,171,232 Shared libraries 235.64% (of 18,320,632)
  +  10,569,349 Package size difference 63.65% (of 16,604,909)

Startup time was also either slightly worse (or the same, within margin of error), when using $(AndroidAotMode)=hybrid. It could likely be due to the larger files being loaded at startup. This diminishes the usefulness of the feature.

alexyakunin commented 2 weeks ago

I would rather prefer to have an opportunity to opt into the interpreter on Android - all depends on whether it's faster than JIT specifically on app startup. And if we assume Profiled AOT works (ok, currently it doesn't, but maybe you'll end up fixing this), the % of methods to interpret should be relatively small.

jonathanpeppers commented 2 weeks ago

I would rather prefer to have an opportunity to opt into the interpreter on Android

@alexyakunin this feels a bit off-topic, but you can do this today in a Release build with:

<PropertyGroup>
  <RunAOTCompilation>false</RunAOTCompilation>
  <UseInterpreter>true</UseInterpreter>
</PropertyGroup>

But I suspect you will see worse performance with this combination. Startup might be OK, but as soon as there is a complicated for-loop, it will perform much slower even than JIT. Additionally, features like hardware intrinsics won't work.

There is no combination of "Normal" AOT mode with interpreter:

https://github.com/dotnet/runtime/blob/91939711895b6141e1be509ee743a8afb7a64ee7/src/native/public/mono/jit/details/jit-types.h#L14-L39

We are consumers of the runtime, so we can only use what they provide.

alexyakunin commented 2 weeks ago

But I suspect you will see worse performance with this combination.

Yes, it is - we tried this option. As far as I remember, it's worse on Android vs even JIT w/o AOT.

alexyakunin commented 2 weeks ago

We are consumers of the runtime, so we can only use what they provide.

@jonathanpeppers What about MONO_AOT_MODE_INTERP and its variants? At glance, it looks like AOT w/o JIT.

jonathanpeppers commented 2 weeks ago

It looks like MONO_AOT_MODE_INTERP is AOT with Interpreter fallback. I discussed in a meeting today, we want to try it out, there are some changes required for it to work.

We'd need to update:

Build time:

https://github.com/dotnet/android/blob/aa668a5c746a7669a145e63efe59d0964093e604/src/Xamarin.Android.Build.Tasks/Tasks/GetAotArguments.cs#L82-L109

Runtime:

https://github.com/dotnet/android/blob/4266c2bc0971a7504c24ca20cbba7a14852fa144/src/native/runtime-base/android-system.cc#L582-L602

alexyakunin commented 2 weeks ago

I got an impression iOS uses these modes, + yeah, there is an incompatibility (well, only partial compatibility) between the modes available in Mono runtime & modes you can specify in Android .targets.

dotnet / android