Support for OS400

[aaronbartell]

There currently isn't jffi support for IBM's OS400\ operating system and I am attempting to build jffi from a git clone of the current master. I see @pierrickrouxel is also working through this issue in the JRuby project and also this issue. Should I be logging this issue here or in the JRuby issues?

**Now known as IBM i or just i. You might also hear it referred to as iSeries.

What I've Done I've made some changes to the build process (see fork) to support OS400 but have come to a point where I need input because the build isn't completing. The config.log declares alloca support was found, but alloca does NOT exist on OS400.

from config.log

. . .
| #define HAVE_ALLOCA_H 1
| #define HAVE_ALLOCA 1
. . .

The stdout to my shell gives more information - specifically:

 [exec] /home/aaron/git/jffi/jni/jffi/LongDouble.c:82:5: error: implicit declaration of function 'alloca' [-Werror=implicit-function-declaration]
 [exec] /home/aaron/git/jffi/jni/jffi/LongDouble.c:82:11: error: incompatible implicit declaration of built-in function 'alloca' [-Werror]
 [exec] cc1: all warnings being treated as errors
 [exec]
 [exec] /home/aaron/git/jffi/jni/GNUmakefile:295: recipe for target '/home/aaron/git/jffi/build/jni/jffi/LongDouble.o' failed
 [exec] gmake: *** [/home/aaron/git/jffi/build/jni/jffi/LongDouble.o] Error 1

[aaronbartell]

I neglected to mention that malloc is supported on OS400.

I see malloc.h is optionally included in configure if _MSC_VER is specified. I also see alloca has the same parameter list/types as malloc. My understanding is that alloca releases memory upon a function going out of scope/returning and malloc requires a manual call to free(ptr) to accomplish the same (Microsoft malloc docs).

I tried altering configure to be as follows (note addition of || _OS400) without success and it produced the same error**.

# ifdef _MSC_VER || _OS400
#  include <malloc.h>
#  define alloca _alloca
# else
. . .

**[exec] /home/aaron/git/jffi/jni/jffi/LongDouble.c:82:11: error: incompatible implicit declaration of built-in function 'alloca' [-Werror]

[headius]

This is the right place to report bugs with jffi!

We can put your good, working changes into a PR any time.

alloca allocates memory from the calling function's stack. When the called function returns, the stack pointer moves back to the caller's position, so the memory is deallocated automatically. This is equivalent to having "char[256] x" in your code.

Because of the stack effects and automatic deallocation, alloca can't directly be emulated with malloc. You're running into problems because the code calling alloca doesn't actually pay attention to the configure results.

I'm not sure how to proceed. We could modify all code that uses alloca, but it seems really unfortunate to cripple other platforms for OS/400. I'm poking around to see if there's a blessed alternative to alloca on OS/400. Maybe you have better developer docs?

[headius]

I think it may be possible to transplant the magic ifdef alloca block from jni/libffi/include/ffi_common.h to jni/jffi/jffi.h. Can you try that?

[penberg]

What compiler does OS/400 use? Does it support C99? If so, you could just try to replace alloca() with variable-length arrays.

[penberg]

If I read the linked compiler documentation correctly, it does support C99.

So you should be able to replace the alloca call with something like this:

diff --git a/jni/jffi/LongDouble.c b/jni/jffi/LongDouble.c
index cb8f243..b6031a7 100644
--- a/jni/jffi/LongDouble.c
+++ b/jni/jffi/LongDouble.c
@@ -32,9 +32,6 @@

 #include <stdio.h>
 #include <stdlib.h>
-#ifdef __sun
-# include <alloca.h>
-#endif
 #include <stdint.h>
 #include <stdbool.h>
 #include <jni.h>
@@ -75,11 +72,10 @@ Java_com_kenai_jffi_Foreign_longDoubleFromString(JNIEnv *env, jobject self, jstr
   jbyteArray array, jint arrayOffset, jint arrayLength)
 {
     long double ld;
-    char* tmp;
     jsize len;

     len = (*env)->GetStringUTFLength(env, str);
-    tmp = alloca(len + 1);
+    char tmp[len + 1];
     (*env)->GetStringUTFRegion(env, str, 0, len, tmp);
     ld = strtold(tmp, NULL);
     jffi_encodeLongDouble(env, ld, array, arrayOffset, arrayLength);

[aaronbartell]

I am using the gcc compiler**. I will work on testing the two suggestions and get back to you.

**

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/QOpenSys/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/4.6.2/lto-wrapper
Target: powerpc-ibm-aix6.1.0.0
Configured with: ../gcc-4.6.2/configure --with-as=/usr/bin/as --with-ld=/usr/bin/ld --enable-languages=c,c++,fortran --prefix=/opt/freeware --mandir=/opt/freeware/man --infodir=/opt/freeware/info --enable-threads --enable-version-specific-runtime-libs --disable-nls --enable-decimal-float=dpd --host=powerpc-ibm-aix6.1.0.0
Thread model: aix
gcc version 4.6.2 (GCC)

[penberg]

@aaronbartell That's pretty recent GCC version. It should support VLAs so if the pragma magic @headius suggested doesn't work, please try the above. You probably need to convert more code, of course, but it should be straight-forward.

[aaronbartell]

@penberg, it made it past LongDouble.c and is now choking on FastNumericInvoker.c so it would seem this is viable. Now I will try suggestion from @headius.

See log here

[aaronbartell]

I think it may be possible to transplant the magic ifdef alloca block from jni/libffi/include/ffi_common.h to jni/jffi/jffi.h. Can you try that?

Here is the log (failed). Below are the diffs so I can make sure I made the correct changes.

jffi.h diff

ffi_common.h diff

[headius]

@aaronbartell I meant to copy the pragmas into jffi.h. libffi and the jffi JNI binding get built separately, and will both need the pragmas. Since libffi built fine without the pragmas and failed (with alloca warnings and related errors) when they were removed, it would seem we're on the right track.

[aaronbartell]

I now have that ifdef code in both areas. I did an ant clean and then ant jar with the shell results found here. Here is the config.log in case that helps.

Let me know if you'd like me to run it again with either -bloadmap or -bnoquiet to obtain more information.

[headius]

That's looking much closer. Something wrong with the final linking of libjffi, but everything seems to have compiled. Perhaps there's a gcc flag needed here to indicate this is a shared library and not an executable?

[penberg]

@headius Yes, -shared. It looks like it's missing from the log @aaronbartell posted.

[aaronbartell]

I believe we have a successful build! See log here. The issue is I had the operating system upper-cased on this line.

I think one thing needing to be cleaned up is the name of the resulting .jar file because it contains a slash (/) in it:

[zip] Building zip: /home/aaron/git/jffi/dist/jffi-ppc-OS/400.jar

I will work to find how to get rid of the slash in the name.

[aaronbartell]

I am now running through the ant test motions. I first changed libtest/GNUmakefile to have a section for os400 by copying the aix equivalent:

diff --git a/libtest/GNUmakefile b/libtest/GNUmakefile
index 9e70664..65c3097 100644
--- a/libtest/GNUmakefile
+++ b/libtest/GNUmakefile
@@ -153,6 +153,13 @@ ifneq ($(findstring mingw, $(OS)),)
   LIBEXT = dll
   PICFLAGS=
 endif
+
+ifeq ($(OS), os400)
+  LIBEXT = a
+  SOFLAGS = -shared -static-libgcc
+  PICFLAGS += -pthread
+endif
+
 ifeq ($(CPU), sparcv9)
   MODEL = 64
 endif

That caused some pthread errors to show up during ant test - see log.

Looking through the libtest/GNUmakefile it doesn't appear PICFLAGS (where -pthread is added) is used on the compiles so I made the following change to see if it would further the test, and it did.

diff --git a/libtest/GNUmakefile b/libtest/GNUmakefile
index 9e70664..65c3097 100644
--- a/libtest/GNUmakefile
+++ b/libtest/GNUmakefile
 $(LIBTEST):  $(TEST_OBJS)
-       $(CC) -o $@ $(LDFLAGS) $(TEST_OBJS) -lm
+       $(CC) -pthread -o $@ $(LDFLAGS) $(TEST_OBJS) -lm

 clean::
        # nothing to do - ant will delete the build dir

Then I run ant test again (note I didn't do ant clean) and seemingly get further, though all of the tests fail (see log here)

Thoughts on what direction I should take these tests?

[headius]

Does anything in jffi/build/test/results show what the errors are? With all tests failing I'd guess it's not able to load libjffi.so.

[headius]

Great progress btw...I'm sure we'll iron this out soon :-)

[aaronbartell]

@headius, there was a file for each failure in jffi/build/test (sorry I missed those). Here is an example of one and they are all conveying basically the same error of java.lang.UnsatisfiedLinkError.

Concerning libjffi.so, here is what a find produces:

-bash-4.2$ pwd
/home/aaron/git/jffi
-bash-4.2$ find . -name libjffi*
./build/jni/libjffi-1.2.a

[aaronbartell]

Doh! If I would have looked further into the aforementioned stack trace I would have seen this:

Caused by: java.lang.RuntimeException: cannot determine operating system

I will do some digging to see what I can find.

[aaronbartell]

I have made it a bit further and am now stuck with errors in this full stack trace.

Here's an excerpt:

java.lang.UnsatisfiedLinkError: /home/aaron/git/jffi/build/jni/jffi-1.2.srvpgm ( 0509-022 Cannot load module /home/aaron/git/jffi/build/jni/libjffi-1.2.srvpgm.so. 0509-026 System error: A file or directory in the path name does not exist.)

It is true the above link/file doesn't exist. What portion of the build should have created it? I ran ant jar and then ant test.

In case you need to know, here are my changes to StubLoader.java to account for OS400:

diff --git a/src/main/java/com/kenai/jffi/internal/StubLoader.java b/src/main/java/com/kenai/jffi/internal/StubLoader.java
index 9a1d842..1e2e14d 100644
--- a/src/main/java/com/kenai/jffi/internal/StubLoader.java
+++ b/src/main/java/com/kenai/jffi/internal/StubLoader.java
@@ -82,6 +82,8 @@ public class StubLoader {
         WINDOWS,
         /** IBM AIX */
         AIX,
+        /** IBM OS400 */
+        OS400,
         /** IBM zOS **/
         ZLINUX,

@@ -136,7 +138,9 @@ public class StubLoader {
         } else if (startsWithIgnoreCase(osName, "sunos") || startsWithIgnoreCase(osName, "solaris")) {
             return OS.SOLARIS;
         } else if (startsWithIgnoreCase(osName, "aix")) {
-            return OS.AIX;
+            return OS.AIX;
+        } else if (startsWithIgnoreCase(osName, "OS/400")) {
+            return OS.OS400;
         } else if (startsWithIgnoreCase(osName, "openbsd")) {
             return OS.OPENBSD;
         } else if (startsWithIgnoreCase(osName, "freebsd")) {
@@ -208,8 +212,10 @@ public class StubLoader {
         if (getOS().equals(OS.DARWIN)) {
             return "Darwin";
         }
+        if (getOS().equals(OS.OS400)) {
+            return "OS400";
+        }

-
         String osName = System.getProperty("os.name").split(" ")[0];
         return getCPU().name().toLowerCase(LOCALE) + "-" + osName;
     }

[headius]

The StubLoader changes look good.

The libjffi.so would be created by the native part of the build. It occurs to me now you may need to manually copy the it to the archive directory (working from memory here since my machine is busted). Then I believe the jar task will incorporate it into the jffi jar file.

[aaronbartell]

Bummer about your machine.

Is this the file you are talking about? Does it matter it is an archive library vs. shared object?

-bash-4.2$ pwd
/home/aaron/git/jffi
-bash-4.2$ find . -name "*libjffi*"
./build/jni/libjffi-1.2.a

[headius]

No, the file is jffi-SOMETHING.jar, and the build will put it in dist:

-build-platform-jar:
     [echo] platform=Darwin
      [zip] Building zip: /Users/headius/projects/jffi/dist/jffi-Darwin.jar

Copy that to archive/ and rebuild, and I believe it will get included into the final jffi jar. The build does not copy to archive/ because the files there are known to git.

[headius]

BTW...the final, working OS/400 native jar, copied to archive/, is what you'd PR (ideally as a separate commit from the changes needed to build it).

[aaronbartell]

Here's the steps I've taken:

1 - rm archive/jffi-ppc-OS400.jar

2 - ant clean

3 - ant jar

4 - mv dist/jffi-ppc-OS400.jar archive/

5 - ant jar <---- This didn't update the archive/jffi-ppc-OS400.jar's timestamp so I figured to continue with another ant clean. I left archive/jffi-ppc-OS400.jar in place from the previous ant jar.

6 - ant clean

7 - ant jar

8 - ant test This produced the previous errors:

java.lang.UnsatisfiedLinkError: could not locate stub library in jar file. Tried [jni/OS400/jffi-1.2.srvpgm, /jni/OS400/jffi-1.2.srvpgm]

and

java.lang.UnsatisfiedLinkError: /home/aaron/git/jffi/build/jni/jffi-1.2.srvpgm ( 0509-022 Cannot load module /home/aaron/git/jffi/build/jni/libjffi-1.2.srvpgm.so.

9 - cp dist/jffi-ppc-OS400.jar archive/

10 - ant test <---- running this produced the same errors as in step 8.

I am guessing I did something wrong. What should I have done?

[headius]

Your steps have a lot of duplication, but the end result should have been correct if the library's getting built right. Once you've built the native archive and copied the jar into archive/ you can just do normal builds from then on. The archive/ jar will not be updated unless you copy a new one in place, but it is where the build gets artfacts to insert into the jffi jar.

Check that the jffi jar has either the OS400 native jar or the files that would be inside it (the native lib itself). Check that the path/filename jffi uses to load that library actually matches the file (I see .so, and you mentioned .a, so there's a lead).

It looks like jffi may not have the right heuristics for building a shared lib filename on OS/400, and so it can't locate the built library.

[aaronbartell]

Here's the archive/jffi-ppc-OS400.jar contents:

The call to System.mapLibraryName(stubLibraryName) produces jffi-1.2.srvpgm. I am not familiar with the .srvpgm extension and am wondering if this is an OS400 thing or if it is a Java thing. OS400 does have something called "service programs" but they have to do with a different/native runtime environment on the machine.

So then I tried manually changing the path to jni/ppc-OS400/libjffi-1.2.a and it still isn't finding it. Other ideas?

[aaronbartell]

A little more info... it seems System.load in StubLoader.java always appends .so to the end of the name. Now I am curious how the AIX folks are getting this accomplished as they also have an archive (i.e. libjffi-1.2.a).

[headius]

I wonder if System.loadLibrary is a way to route around this. I've never spent a lot of time looking at how those paths work.

I've put out a call for help on Twitter. If System.load is actually appending an invalid extension (and System.mapLibraryName is adding a bogus or atypical extension) that would seem to be a bug in IBM's JVM. That seems unlikely to me.

Do you have an IBM iSeries or J9 support person you can talk to about this? I'm running out of ideas and don't have an iSeries machine here to play around on :-)

[aaronbartell]

I tried swapping in System.loadLibrary (replacing System.load) and have the same results.

that would seem to be a bug in IBM's JVM. That seems unlikely to me.

Agreed. Unlikely, though I don't know how common it is to load C/C++ archive files into Java on this platform.

Do you have an IBM iSeries or J9 support person you can talk to about this?

I do have a person at IBM I can reach out to but I wanted to ask one more question first. Do you know where the loadJar() process/method is looking for the jar file? Here is what I have for jars in the entire git clone:

-bash-4.2$ find . -name "*.jar"
./archive/jffi-Darwin.jar
./archive/jffi-arm-Linux.jar
./archive/jffi-i386-FreeBSD.jar
./archive/jffi-i386-Linux.jar
./archive/jffi-i386-OpenBSD.jar
./archive/jffi-i386-SunOS.jar
./archive/jffi-i386-Windows.jar
./archive/jffi-ppc-AIX.jar
./archive/jffi-ppc-Linux.jar
./archive/jffi-ppc64-Linux.jar
./archive/jffi-s390x-Linux.jar
./archive/jffi-sparc-SunOS.jar
./archive/jffi-sparcv9-SunOS.jar
./archive/jffi-x86_64-FreeBSD.jar
./archive/jffi-x86_64-Linux.jar
./archive/jffi-x86_64-OpenBSD.jar
./archive/jffi-x86_64-SunOS.jar
./archive/jffi-x86_64-Windows.jar
./archive/jffi-ppc-OS400.jar    <----------------------
./lib/junit/junit-3.8.2.jar
./lib/junit_4/junit-4.5-src.jar
./lib/junit_4/junit-4.5.jar
./build/native.jar
./dist/jffi.jar
./dist/jffi-ppc-OS400.jar      <----------------------------
./dist/jffi-complete.jar

I'm running out of ideas and don't have an iSeries machine here to play around on :-)

My hope is to get a chroot environment setup on a community IBM i so CI (like travis-ci.org) can be run. The price points for "IBM i in the cloud" are coming down\ but still a decent amount of work to setup. If I get one setup I will let this group know.

\ $100/month from iDevCloud.com for a dedicated virtual machine used for educational/non-commercial purposes

[headius]

Do you know where the loadJar() process/method is looking for the jar file?

The contents of all the archives are re-packed directly into the eventual jffi jar, I believe. So if you look in the jffi jar, you should see the native libs for all platforms.

If I get one setup I will let this group know.

That would be great :-) I also poked a few friends of mine at IBM who might be able to assist us getting this library built and loading properly.

[aaronbartell]

So if you look in the jffi jar, you should see the native libs for all platforms.

Here is what I have in the dist/ directory after running a build:

$ ls dist/
jffi-complete.jar   jffi-ppc-OS400.jar  jffi.jar

The following lists the contents of each .jar file. As you can see, only jffi-complete.jar and jffi-ppc-OS400.jar have the .so and .a files for each platform. Should jffi.jar also have them?

dist/jffi-complete.jar

dist/jffi-ppc-OS400.jar

dist/jffi.jar

[headius]

FYI, we'd prefer if you used Gist for long pastes.

Right right, sorry...I meant the jffi-complete jar. If you were using jffi.jar directly, you could try using jffi-complete.jar instead (maybe you already did).

Let me confirm I can run ant test here and then I'll confirm it's using the complete jar. If it is, we still have some work to do on loading the OS/400 binary.

[headius]

Ok...so ant test works for me just fine on OS X, after removing -Werror from libtest's makefile.

This line in build.xml indicates where it's looking for the native libs:

<sysproperty key="java.library.path" value="${build.native.dir}"/>

build.native.dir here is /jni. If you have the appropriate library there, the test should work...if it's able to load the file correctly.

I wonder if you could try renaming the library to the extension jffi is looking for...at least then we could tell if the library works, and focus on getting it to load properly.

[aaronbartell]

FYI, we'd prefer if you used Gist for long pastes.

Will do. I've updated my previous post with gist entries to clean it up for historical purposes.

I wonder if you could try renaming the library to the extension jffi is looking for...at least then we could tell if the library works, and focus on getting it to load properly.

I renamed build/jni/libjffi-1.2.a to build/jni/libjffi-1.2.srvpgm.so. Then the error said it was looking for ./test.srvpgm so I renamed ./libtest.a to ./test.srvpgm (it wasn't looking for a .so extension). Renaming those got us past the loading issue and produced this java dump and this file: javacore.20140909.151537.67913.0002.txt.

It appears line 982 is where it first references the Java side of jffi.

Snap.20140909.151537.67913.0003.trc (binary, let me know if I should jextract it)

core.20140909.151537.67913.0001.dmp (2.2GB bytes in size so I didn't create a gist)

The rename obviously worked for loading the files, but do these errors tell us the original .a files weren't built correctly or is it telling us we need to build them initially as .so?

[headius]

Wow that's a cool error... illegal instruction!

My first thought was that it's related to some assembly-generation logic in JNR that tries to make faster-than-jni stubs on the native side, but that logic all lives in jnr-ffi.

So then the real answer seems to be that .a isn't the output format we want to use. I poked around a bit, and at least on Linux a .a file is intended to be used as a statically-bound library, not a dynamically-bound library. I feel like there's make/cc flags missing still.

[DanHeidinga]

FYI - my iSeries contacts have confirmed that iSeries JVM can recognize and load ".so", ".a" & ".srvpgm" files.

They showed a test case that loads all three kinds of libraries, provided the native library is on the java.library.path (which from the native stacktrace in the javacore, it looks like it is).

[aaronbartell]

I've done some more looking and found the JNIEXT and LIBEXT settings in jni/GNUmakefile and libtest/GNUmakefile respectively. I altered them, as shown below, to not resort to .a extension and instead .so. That resulted in this successful build with .so as extension but the unit tests still fail with the illegal instruction (same as when I manually renamed from .a to .so.

Just now saw response from @DanHeidinga. Can we see the example code IBM showed or was it done under an NDA?

diff --git a/jni/GNUmakefile b/jni/GNUmakefile
index bdf2fe1..b200332 100755
--- a/jni/GNUmakefile
+++ b/jni/GNUmakefile
@@ -223,7 +223,7 @@ ifeq ($(OS), os400)
   SOFLAGS = -shared -static-libgcc
   CFLAGS += -pthread
   LDFLAGS += -pthread
-  JNIEXT = a
+  #JNIEXT = a
   STRIP = strip
 endif

diff --git a/libtest/GNUmakefile b/libtest/GNUmakefile
index 9e70664..0e60019 100644
--- a/libtest/GNUmakefile
+++ b/libtest/GNUmakefile
@@ -130,6 +130,12 @@ ifeq ($(OS), solaris)
   SOFLAGS = -shared -static-libgcc
 endif

+ifeq ($(OS), os400)
+  #LIBEXT = a
+  SOFLAGS = -shared -static-libgcc
+  PICFLAGS += -pthread
+endif
 $(LIBTEST):  $(TEST_OBJS)
-       $(CC) -o $@ $(LDFLAGS) $(TEST_OBJS) -lm
+       $(CC) -pthread -o $@ $(LDFLAGS) $(TEST_OBJS) -lm

[headius]

@DanHeidinga Can we get an example of how to properly make a JNI library on iSeries? cc command line, etc?

[headius]

@aaronbartell Could you look to see whether libraries under JAVA_HOME are .a or .so? Several of them would be JNI extensions; if we don't see .a files then we may be missing something about how to build this stuff.

[aaronbartell]

@headius I ran the below find and here is the output.

bash-4.2$ find /QOpenSys/QIBM/ProdData/JavaVM/jdk60/32bit -name "*.a" -o -name "*.so"

I started digging through IBM documentation for JNI calls to C/C++ and found Getting started with Java native methods. Then I found this page with a full example of JNI to C.

Here are the example source files: PaseExample1.h PaseExample1.c PaseExample1.java

I then used the following to get the example running:

javac PaseExample1.java
javah -jni PaseExample1
export LIBPATH=/home/aaron
gcc -c -fPIC PaseExample1.c -o PaseExample1.o -I/QOpenSys/QIBM/ProdData/JavaVM/jdk60/32bit/include/
gcc -c PaseExample1.c -o libPaseExample1.so -I/QOpenSys/QIBM/ProdData/JavaVM/jdk60/32bit/include/
gcc PaseExample1.o -shared -o libPaseExample1.so
java PaseExample1.java

Output

-bash-4.2$ java PaseExample1
Value of str is 'String for PaseExample1'

Thoughts?

For historical purposes I've learned why it was looking for .srvpgm extension on this page.

[DanHeidinga]

We use xlC rather than gcc on iSeries so the commandlines may not be that helpful. That being said, here's the commandline used to build one of the internal jni argument test libraries:

xlC_r -O3 -DIPv6_FUNCTION_SUPPORT -q32 -q mbcs -qlanglvl=extended -qarch=ppc -qinfo=pro -qalias=noansi -qxflag=LTOL:LTOL0 -qsuppress=1506-1108 -D_XOPEN_SOURCE_EXTENDED=1 -D_ALL_SOURCE -DRS6000 -DAIXPPC -D_LARGE_FILES -qtbtable=full -qlist -qsource -DJ9OS_I5 -DJ9OS_I5_V7R2 -I..//iseries -qhalt=w -I. -I../include -I../include_core -I../oti -I../nls -qtbtable=full -qlist -qsource -DJ9OS_I5 -I/osxpf/v7r2m0f.cuep/bld/cmvc/pase.pgm/p5.cuep/include -I/osxpf/v7r2m0.cuep/bld/cmvc/pase.pgm/p5.cuep/include -I/afs/rchland.ibm.com/usr8/j9/src/rebuilt/jdk70/32bit27/sr1fp1.v7r2/jre/lib/ppc/default/iseries -DUT_DIRECT_TRACE_REGISTRATION -c -o cdefs.o cdefs.c

rm -f ../libjniargtests.so

ld -b32 -G -bnoentry -bernotok -bmap:jniargtests.map -bE:jniargtests.exp -bI:..//iseries/i5exports.exp \ -bI:/osxpf/v7r2m0.cuep/bld/cmvc/pase.pgm/p5.cuep/lib/as400_libc.exp -bI:/afs/rchland.ibm.com/usr8/j9/src/rebuilt/jdk70/32bit27/sr1fp1.v7r2/jre/lib/ppc/default/iseries/i5exports.exp -L. -L../ -L../lib/ -o ../libjniargtests.so\ copyright.o args_01.o args_02.o args_03.o args_04.o args_05.o args_06.o args_07.o args_08.o args_09.o cdefs.o \ -lc_r -lC_r -lm -lpthread

[headius]

@aaronbartell Were you able to make any more progress based on @DanHeidinga's response? We're so close!

Your find output seems to indicate that we probably do need .so for the JNI libraries, since e.g. libhprof is also a JNI library.

[aaronbartell]

I've cleaned up a few more things so my build/test wasn't such a hack and better adheres to how OS400 loads .a files. Latest changes here for anyone that is interested.

Going back to to @DanHeidinga's input response.

Here is the IBM xlc compiler options list (expand left nav to see options). Here is the Option mappings for gxlc and gxlc++ utilities - XL C/C++ Enterprise Edition V8.0 for AIX which we can use to learn the gcc to xlc option mappings.

The majority of the mappings don't have a direct translation. My talents are lacking regarding what options are significant. Could anybody else give input on that front? Here is the latest successful build results. Here is the latest ant test Java dump. Note that libjffi-1.2.a is referenced but it doesn't appear to give a line number. Is there a gcc setting that would provide better errors?

[pierrickrouxel]

Hi @aaronbartell. I tried to help you, but this work is totally outside my area of ​​expertise. I hope you will come to complete this task. I am extremely grateful to you for this library that will dramatically improve my use of JRuby.

[pierrickrouxel]

I did some compilation tests. I do not understand why you absolutely want to use xlc. Originally there is no gcc or xlc installed on the machine. What makes you force the use of one rather than another?

[aaronbartell]

@pierrickrouxel , the xlc compiler entered the picture as a way to learn what options we may need to specify on gcc. I would much prefer that gcc was used so others can also easily compile JRuby without the need to purchase the xlc compiler (not an in-expensive purchase).

I did find that perzl.org has a libffi port for AIX and the rpm installs successfully on IBM i. I am digging into that to see if I can simply use the compiled libffi files from there and plop them into .../jffi/build/jni/libffi-ppc-os400/.libs/

[pierrickrouxel]

Oh ! Ok I misunderstood. I had not seen that the tests did not pass. Sorry.

[headius]

@aaronbartell Any recent progress? I just released 1.2.8, but it would sure be nice to have OS/400 support in 1.3.0 (which will require rebuilds of a number of other native libs anyway).

[aaronbartell]

@headius Status: I am working on this again. I've started documenting the entire IBM i build process on my jffi fork in hopes others can help, though I am also working to get an IBM i machine provisioned so you (or others) can ssh in and poke around to speed things up. Stay tuned.

Progress: I've done an upstream merge into my fork and the current build is yielding these results. In short, the error is as follows:

     [exec] In file included from /home/abartell/git/jffi/build/jni/libffi-ppc-os400/include/ffi.h:67:0,
     [exec]                  from /home/abartell/git/jffi/jni/jffi/jffi.h:45,
     [exec]                  from /home/abartell/git/jffi/jni/jffi/Array.c:39:
     [exec] /home/abartell/git/jffi/build/jni/libffi-ppc-os400/include/ffitarget.h:159:5: error: "_CALL_ELF" is not defined [-Werror=undef]
     [exec]  #if _CALL_ELF == 2

Here is file ffitarget.h. Note, I have not encountered this _CALL_ELF error before and am now looking into it.

UPDATE 2015-04-16 12:29pm CST I commented out that section of ffitarget.h and now it is building successfully and I have file dist/jffi-ppc-OS400.jar. I will go back through this conversation on what to do next.

UPDATE 2015-04-16 3:40pm CST I am now getting the following error when running ant -v test (full log). I don't know which class it is looking for. Is it looking for the unit test files in build/test/classes/com/kenai/jffi?, because those do exist. Is it looking for jffi-ppc-OS400.jar?, because I copied it to archive/ after the most recently successful ant jar.

    [mkdir] Created dir: /home/abartell/git/jffi/build/test/results
    [junit] Exception in thread "main" java.lang.NoClassDefFoundError:
    [junit] Caused by: java.lang.ClassNotFoundException:
    [junit]     at java.net.URLClassLoader.findClass(URLClassLoader.java:666)
    [junit]     at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:942)
    [junit]     at java.lang.ClassLoader.loadClass(ClassLoader.java:869)
    [junit]     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:336)
    [junit]     at java.lang.ClassLoader.loadClass(ClassLoader.java:847)
    [junit] Running com.kenai.jffi.Batch-With-Multiple-Tests
    [junit] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec
    [junit] Tests FAILED (crashed)

Here's the contents of build/test/results/IGNORETHIS.txt - the only file in the results/ directory. Note the line Testcase: null which leads me to believe it isn't find the build/test/classes/com/kenai/jffi files. Thoughts?

Testsuite: com.kenai.jffi.Batch-With-Multiple-Tests
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec

Testcase: null took 0.007 sec
    Caused an ERROR
Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.
junit.framework.AssertionFailedError: Forked Java VM exited abnormally. Please note the time in the report does not reflect the time until the VM exit.