wasabii
commented
2 years ago Where my thinking is at now:
1) Turn the magic-JNI-stuff into an explicit system. Add an argument to ikvmc named -externs:AssemblyName. This turns into an Attribute on the generated assembly. During compilation, native methods are turned into calls to a delegate retrieved from IKVM.Runtime. So, static native void foo() turns into a static private field named <>native_foo or something, with a delegate type that matches it's signature. Code is inserted at the beginning of the static ctor to initialize these fields by calling into IKVM.Runtime. Something like IKVM.Runtime.JVM.CreateExternDelegate(MethodInfo). This call looks for the ExternsAttribute or whatever, and uses it to determine which assembly the externs are in. That assembly is scanned for the appropriate static method, and a delegate is built. The 'native' method is turned into a call to this delegate.
This makes the magic JNI stuff less magic, and breaks a cycle. IKVM.Java will no longer have a hard dependency on whereever the externs live.
2) Move the IKVM.Java externs out of IKVM.Runtime into IKVM.Java.Externs. The build should now be capable of building IKVM.Java BEFORE building IKVM.Java.Externs, since the binding is done at runtime. IKVM.Java.Externs has a hard reference to IKVM.Java. This allows the externs library to directly access types in IKVM.Java, but avoids IKVM.Java doing the same in reverse. And since all of the externs are moved out of IKVM.Runtime, a significant chunk of the cycle is broken.
3) The remaining calls from IKVM.Runtime to IKVM.Java should now be for built-in or special types. Object, String, etc. These can be handled with a reflection-based trampoline of some kind. That is, IKVM.Runtime no longer directly access java.lang stuff. Instead, it has to access all of the methods through a proxy class. It can still receive the objects, as type 'object'. And the primitive types work fine. But it can't directly touch anything else. IKVM.Runtime can now fully build before IKVM.Java builds.
At this point we should be able to build IKVM.Runtime first, before building any OpenJDK components. IKVM.Runtime-ref can go away. IKVM.Java can build second, and can directly access IKVM.Runtime. IKVM.Java-ref goes away because IKVM.Runtime doesn't need it.
IKVM.Java.Externs builds third. Referencing both IKVM.Runtime and IKVM.Java.
Currently the IKVM project requires a number of circular references between assemblies. IKVM.Java references IKVM.Runtime. IKVM.Runtime references IKVM.Java. IKVM.Runtime.JNI is referenced by ... etc.
These are solved in the new build system (and old) with dummy projects suffixed with the name '-ref' (as they are like, "reference assemblies" but not really). These include source files from each other, knock out some of the bodies with #ifdefs, or in the case of IKVM.Java-ref, simply include a bunch of empty classes to function as "signatures".
This is non-ideal. It works. It's not TERRIBLE. But it's not ideal. It makes understanding the build process harder than it needs to be. And it makes it more complicated than it needs to be. For instance, projects can't just reference what they depend on, and expect Copy Local to work right. They have to reference their -ref counterparts, and ensure Copy Local is false, and then have additional projects at the top of the dependency stack that include the non-ref counterparts. For instance, IKVM-pkg, which builds the NuGet package.
Intellisense in VS is also completely hosed due to this.
It's just not great.
There are two broad categories of circular references.
a) The "JNI" backed native methods that implement the functionality required by Java classes in OpenJDK, but which live in IKVM.Runtime. There's a lot of cross talk here. Java code might declare a 'native' method, which is usually JNI, but have the static compiler rewrite that to a call to a magic static class in IKVM.Runtime. And then that static class may obtain references from IKVM.Java itself, such as the related type, or other types, and call back and forth. b) The Runtime references the IKVM.Java assembly to have available to it some strongly typed base classes, like Object, String, etc.
I propose working to break these links one way. IKVM.Runtime should not be accessing IKVM.Java directly.
For (a), In the regular OpenJDK library, these native methods are actual native methods, actually invoked with actual JNI. As such, they are passed a JNIEnv*, and do all their callbacks and object lookups by using the JNI interface. As such, they're late bound. IKVM is early bound. There would be nothing wrong by just doing this the same way OpenJDK itself does it. It is nice to be able to directly reference Java types back and forth, but it's not strictly necessary. To alleviate this, I would suggest we define a Managed JNI interface to take it's place. Constructed to model the standard JNI interface, .NET methods implementing "native" functionality would have a JNIEnv-like class passed to them, which they can use to reach back into the Java code. We have a lot of things available to us here, so it would not be as terrible as C-style JNI. We can build Delegates on the fly. We can make them fast. We can cache them. We can even make them type-safe.
Said "managed JNI" interface could potentially be made public and exposed to users. This might help those cases where somebody is attempting to convert a Jar file which uses JNI to a real native library, but they think they can get away with replacing the native functionality with managed code. Redirect a call to System.loadLibrary to load a "managed JNI thing", then call native methods like normal.
For (b), I would suggest that the Universe (or it's future replacement) have a phase where it binds to the core Java types by name, and as such doesn't need to directly access them. It can probably use the same JNI-ish interface we define for (a), but with an upfront preload stage that finds IKVM.Java late-bound.
IKVM.Java however would be free to access IKVM.Runtime directly, either by directly accessing methods within it using @cli.IKVM.Runtime, or by using the previously mentioned 'managed JNI' interface.
Just Thoughts.