DSO handling for like-named libraries is broken

So, I've been pondering how to fix this issue on our end and I don't see a way to get rid of the ambiguity involved here. Here's what's going on:

We have two input items:

SkiaSharp.dll, the managed library
libSkiaSharp.so, native library used by the managed one

At build time, the AOT compiler adds a third item, the AOT image generated from SkiaSharp.dll, which is written to file named SkiaSharp.dll.so and packaged by us under the libaot-SkiaSharp.dll.so name. As part of the build, we mutate the input shared library name and generate an xxHash for each variation. Then, at the runtime, when MonoVM asks us to load a native library, we calculate the xxHash and look it up in a table generated at build time. This works very well, except in this scenario we have an ambiguity:

        %struct.DSOCacheEntry {
                i64 u0x12e73d483788709d, ; from name: SkiaSharp.so
                i64 u0x3cb282562b838c95, ; uint64_t real_name_hash
                i1 false, ; bool ignore
                ptr @.DSOCacheEntry.23_name, ; name: libaot-SkiaSharp.dll.so
                ptr null; void* handle
        }, ; 71
        %struct.DSOCacheEntry {
                i64 u0x12e73d483788709d, ; from name: SkiaSharp.so
                i64 u0x43db119dcc3147fa, ; uint64_t real_name_hash
                i1 false, ; bool ignore
                ptr @.DSOCacheEntry.7_name, ; name: libSkiaSharp.so
                ptr null; void* handle
        }, ; 72

Input names of SkiaSharp.dll.so and libSkiaSharp.so were at some point mutated to the same form of SkiaSharp.so - a reasonable mutation, because managed code may refer to libSkiaSharp.so using the [DllImport ("SkiaSharp")] attribute and so MonoVM will asks us to load SkiaSharp.so. However, the Mono JIT will at some point load the SkiaSharp.dll library and try to find its AOT image, resulting in request to load SkiaSharp.so as well! This time, however, MonoVM wants to find what we have packaged as libaot-SkiaSharp.dll.so and we have a conflict.

Putting aside the fact that the lookup table should not have any entries with the same hashes, we have a decision to make when MonoVM asks us to dlopen a library named SkiaSharp.so. There are two contexts in this case:

We're being asked to do so as part of the P/Invoke mechanism. This is, essentially, not ambiguous since we know Mono wants the "real" library, not the AOT image
The dlopen hook in our runtime is invoked by MonoVM to load the shared library, but we have no idea what's the purpose - it can refer both to libSkiaSharp.so (for some reason) and to libaot-SkiaSharp.dll.so and we can't resolve that ambiguity without more context.

The immediate solution, and one which I'm going to implement now, is to completely ignore libaot-SkiaSharp.dll.so when building the app - we won't package it, we won't load it when Mono JIT asks for its AOT image. Instead, we will in this case load the libSkiaSharp.so library, and once this dotnet issue is fixed, MonoVM's JIT will simply ignore the library, finding that it's not an AOT image. This is reasonable, because the application can work just fine without the AOT image loaded, but it cannot work without libSkiaSharp.so loaded (all the p/invokes would fail).

The better solution is to avoid the ambiguity. In order to do that, I think the best way would be if Mono JIT asked to load aot-Assembly.so when it loads Assembly.dll and looks for its AOT image - this could be implemented only on Android or it could be used on all the platforms. Furthermore, AOT image loader should then refrain from asking for other permutations on the name, i.e. it should not follow with the attempt to load Assembly.so etc.

/cc @lambdageek

dotnet / android

DSO handling for like-named libraries is broken #9081

Android framework version

Affected platform version

Description

Steps to Reproduce

Did you find any workaround?

Relevant log output