gpu
commented
8 years ago Thanks for this contribution!
Actually, I created an Android version of JOCL ~4 years ago, but I had a non-disclosure-agreement with some company (which now does no longer seem to give a ... care about this topic - and the NDA was limited to 3 years, so ... the risk of being sued should be rather small). I remember that there had been some other issues. From the tip of my head, there was something related to the inheritance hierarchy of the Buffer class, but I'll have to revise this - it might be that this is no longer an issue.
I'll have to read through the changes in more detail. I'll try to do this during the weekend, and give it a try early next week (I should even be able to test it on some android device - the above mentioned company provided me one, and they never wanted it back...).
But quickly going over the changes and the points that you mentioned:
- Support for Android ARM architectures: Should be fine
- Sizeof.computePointerSize(): The
"sun.arch.data.model"string is, according to the name, likely just a "legacy" property and too specific. But according to https://docs.oracle.com/javase/tutorial/essential/environment/sysprop.html ,System.getProperty("os.arch");should be a portable approach, and I'd prefer this to a JNI based solution (if it works on Android as well) - LibInitializer.java : These hard-coded paths look a bit odd, but from a quick websearch, "trying out different names" seems to be the way to go here. (This might be related to the last TODO, see below)
- Android shared library outputs to nativeLibraries/${ANDROID_ABI}/ : I'm not entirely sure (yet) what this is about (and have I'll have to revise how I handled the native library in my first tests). Usually, the intention in JOCL is to pack the native library into the JAR, unpack it at runtime, and load it with
System.load. TheSystem.loadLibrarypath is mainly for local tests and as a fallback. However, there are some open questions, also regarding Maven and the deployment of the natives (also see the TODOs below) - Document building for Android: During my first tests, I used hand-crafted
.mkfiles to be fed into the NDK - it indeed was painful. And I thinkandroid-cmakedid not exist back then - I'll have a closer look at this.
Regarding the TODOs:
Generalizations regarding the target platforms (and their build processes):
These are always welcome. However, they may increase the complexity of the makefiles considerably. (You might have noticed that the current issues in the issue tracker are related to "embarassingly basic" problems of building on other platforms - I hope that a robust, portable and maintainable solution can be found there soon...)
The native library deployment (and loading) is indeed an issue. There have been some related discussions in the JOCL forum. The migration of JOCL from "a downloadable ZIP" to "a Maven artifact" was a hassle. (For me, because I'm not a Maven expert, and ... native libraries are always difficult in Maven).
I'll try to summarize some points here (and consider to open one or more dedicated issues in the issue tracker for these) :
- The current process is to build the native libraries (basically all of them), place them into the native libraries folder, and let them be packed into the JAR by Maven. At runtime, the matching native library is unpacked, and loaded with
System.load(notSystem.loadLibrary). This has the drawback that all native libraries must be available before creating the Maven artifact. The advantage is that one can just download and drop the JAR into the classpath and it should run everywhere. - An alternative to the "One-JAR-With-All-Natives" approach would be to have a plain, pure Java JAR, and one JAR for each native library. These Native-JARs could then be declared as (profile-dependent) dependencies for the main JAR, and downloaded from Maven Central as they are required. The possible advantage that I see here would be that one could add the native parts individually. (Although I like the "one-JAR" solution, I'm not sure how well this would play with Android). There are other projects (JOGL, LWJGL...) that may provide some "inspiration" here.
- (There are things like the https://github.com/maven-nar/nar-maven-plugin that should simplify the build process - but in the end, each native library has to be compiled on a different platform - so I don't see such a great benefit here)
- The
LibUtilsclass has evolved over time, and generalized step by step, to allow more flexibility in the library loading. Recently, I added the option to load dependencies before loading the actual JNI library (mainly for JOCLBLAS and JOCLBlast). However, it is still lacking some features that are already on my TODO list. Mainly, a proper versioning scheme. It should more easily be possible to handle different versions of the natives. Some "version number string" (with semantic versioning pattern) should likely be integrated here. - For the case of OpenCL, there is a special caveat: The JNI library does not link directly against the OpenCL implementation library. It links against the
OpenCL.DLL, which contains the ICD (here is a recent related forum thread). In the end, this is the reason for theLibInitializercontortions, which would not be necessary for "simple" JNI bindings. - Of course, I already considered to extract the
LibUtilsinto a dedicated project. (I also hate the redundancy between JOCL and JCuda, and when more features are integrated (version numbers, dependecy handling...), this becomes an even more pressing issue). But there already are approaches for this - for example, https://github.com/scijava/native-lib-loader . I have to evaluate this and compare its functionality to theLibUtils, to see whether this library could replace theLibUtils, or whether theLibUtilsshould indeed become a dedicated project.
(BTW: I didn't even know BIDMat, thanks for this pointer)
phlip9
nativeLibraries/${ANDROID_ABI}/to comply with canonical Android lib location (also the only place the Android runtime checks when searching for a shared library through System.loadLibrary(...)).TODO: More general cross-compiling support?
TODO: Better, more explicit support for other OS's / ARCH's?
TODO: Library would be more accessible if uploaded to maven repo. But we also have native libraries which makes everything painful.
TODO: Refactor out native library loading to another library? There's lots of redundancy and slight changes between this LibUtils, jcuda's LibUtils, our LibUtils (BIDMat), etc...