search

jpype-project / jpype

JPype is cross language bridge to allow Python programs full access to Java class libraries.
http://www.jpype.org
Apache License 2.0
1.12k stars 181 forks source link

Unable to use JPype on IBM I series #719

Open Devender2806 opened 4 years ago

Devender2806 commented 4 years ago

Dear Team,

We are attempting to install python 3.6 64 bit on IBM I server (PACE,AIX) so that we can connect to Oracle database. I started to work with JDBC thin driver concept first hence downloaded and installed jaydebeapi and JPype1 using the pip3 install command. The libraries installed fine, but when i initiate a connection i get core dump. I tried reading the core file with java tools and all it tells me is no JRE.

I can see java and javac executable files in the one of the directories and when i type java or javac it displays list of options. I also tried giving the path of libjvm.so file but doesn't help. I setup the JAVA_HOME and LIBPATH and PATH variables accordingly but i cannot get past the core dump. We have almost 7 JVMs running on the I series box currently.

Thrameos commented 4 years ago

Unless you send one with the start up then there is nothing to be done. Please execute the tests as requested so that I get usable diagnostics.

Devender2806 commented 4 years ago

Please see the trace from the hello world code.

trace_hello_world.txt

Devender2806 commented 4 years ago

Just sent the trace from hello world code.

On Fri, May 1, 2020 at 1:54 PM Karl Nelson notifications@github.com wrote:

Unless you send one with the start up then there is nothing to be done. Please execute the tests as requested so that I get usable diagnostics.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622515955, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6Q5SMUXD4L4CTGZQ43RPMLHPANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

So the code is failing in...

       virtual void loadLibrary(const char* path) override
        {
                JP_TRACE_IN("LinuxPlatformAdapter::loadLibrary");
#if defined(_HPUX) && !defined(_IA64)
                JP_TRACE("shl_load", path);
                jvmLibrary = shl_load(path, BIND_DEFERRED | BIND_VERBOSE, 0L);
#else
                JP_TRACE("dlopen", path);  
                jvmLibrary = dlopen(path, RTLD_LAZY | RTLD_GLOBAL);  // <====== Returned NULL
#endif // HPUX

                if (jvmLibrary == NULL)
                {
                        JP_TRACE("null library");
   // INSERT CODE HERE

So next we need to add a line to figure out the problem....

https://www.ibm.com/support/knowledgecenter/ssw_aix_71/d_bostechref/dlopen.html

It looks like we need to add two diagnostic calls I would recommend

          JP_TRACE("ErrNo" errno);
          if (errno == ENOEXEC)
          {
             JP_TRACE(dlerror());
          }
          else
          {
               JP_TRACE("No diagnositics");
          }

Rebuild (fix any imports by adding #include <errno.h> to the top if needed). The run the hello world again and send the log.

Devender2806 commented 4 years ago

so is the change to the code ? if (jvmLibrary == NULL) { JP_TRACE("null library"); JP_TRACE("ErrNo" errno); if (errno == ENOEXEC) { JP_TRACE(dlerror()); } else { JP_TRACE("No diagnositics"); }

Thrameos commented 4 years ago

Yes insert the diagnostic code and try again.

Devender2806 commented 4 years ago

if (jvmLibrary == NULL) { JP_TRACE("null library"); // JP_RAISE_OS_ERROR_UNIX( errno, path); // GCOVR_EXCL_LINE JP_TRACE("ErrNo" errno); if (errno == ENOEXEC) { JP_TRACE(dlerror()); } else { JP_TRACE("No diagnositics");

                    }
            JP_TRACE_OUT;
    }

is the intend correct ? and i commented // JP_RAISE_OS_ERROR_UNIX( errno, path), iam getting this error

Devender2806 commented 4 years ago

I think i found the issue..re-running it

Devender2806 commented 4 years ago

getting this error In file included from native/common/include/jpype.h:191:0, from native/common/jp_platform.cpp:17: native/common/jp_platform.cpp: In member function 'virtual void LinuxPlatformAdapter::loadLibrary(const char*)': native/common/jp_platform.cpp:121:42: error: expression cannot be used as a function JP_TRACE("ErrNo" errno); ^ native/common/include/jp_tracer.h:35:39: note: in definition of macro 'JP_TRACE'

define JP_TRACE(...) JPTracer::trace(__VA_ARGS__)

On Fri, May 1, 2020 at 3:22 PM Karl Nelson notifications@github.com wrote:

Yes insert the diagnostic code and try again.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622551204, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6W65AABGQ6XFJ6XJ2LRPMVQ3ANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

Sorry missing a comma...JP_TRACE("ErrNo", errno);

Devender2806 commented 4 years ago

ok i changed it "ErrNo",errno) and it worked

On Fri, May 1, 2020 at 3:22 PM Karl Nelson notifications@github.com wrote:

Yes insert the diagnostic code and try again.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622551204, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6W65AABGQ6XFJ6XJ2LRPMVQ3ANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

This is the trace after latest build trace_after_diagnosis.txt

Thrameos commented 4 years ago

ErrNo 2 would be no such file or directory. Please check to see if the libjvm.so file is there and readable.

Devender2806 commented 4 years ago

I see /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/j9vm/libjvm.so and it has read access for User group and others, but this is not my JAVA_HOME setup, JAVA_HOME is set to the JDK directory, should i include this in the PATH ?

On Fri, May 1, 2020 at 3:55 PM Karl Nelson notifications@github.com wrote:

ErrNo 2 would be no such file or directory. Please check to see if the libjvm.so file is there and readable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622564750, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SVSKOPNSYZLYQ2QLTRPMZNVANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

looks like the code is looking at /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/compressedrefs/libjvm.so

Thrameos commented 4 years ago

If you want to try a different JVM just place it as the first argument to the startJVM call.

  jpype.startJVM('/QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/j9vm/libjvm.so')

But here is the important part. If the JVM has shared libraries that it depends on, then those need to be in the PATH or LIBPATH. Unfortunately, as I can't see your system I can't tell what libraries are required. On some machines this will be $JAVA_HOME/bin, on others it may be $JAVA_HOME/lib. The settings depend on the platform, and as I do not have access to any AIX systems there is no way for me to know.

Devender2806 commented 4 years ago

Can you tell me what specific files to look for?

On Fri, May 1, 2020, 17:24 Karl Nelson notifications@github.com wrote:

If you want to try a different JVM just place it as the first argument to the startJVM call.

jpype.startJVM('/QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/j9vm/libjvm.so')

But here is the important part. If the JVM has shared libraries that it depends on, then those need to be in the PATH or LIBPATH. Unfortunately, as I can't see your system I can't tell what libraries are required. On some machines this will be $JAVA_HOME/bin, on others it may be $JAVA_HOME/lib. The settings depend on the platform, and as I do not have access to any AIX systems there is no way for me to know.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622592006, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6XC2NHMBYHTFWTI4JTRPND3JANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

The tools I use for such a task are somewhat platform dependent. Usually nm, objdump and strings can be used to figure out what libraries are required for a shared library. If those libraries are not on the system library path then I would add them. Also I will often just write a short piece of test code the exercises the single call that I am trying to get working rather than trying to work in a large framework. Once I get the call working (in this case dlopen) then I port the required changes back to code.

I will try to see if I can get you a better answer, but you likely need to consult an aix expert.

Devender2806 commented 4 years ago

is this error related to JVM as well ? LinuxPlatformAdapter::getSymbol: EXCEPTION THROWN 0x9001000a05dcd18 Unable to load symbol [JNI_CreateJavaVM], error = Invalid argument

On Fri, May 1, 2020 at 5:24 PM Karl Nelson notifications@github.com wrote:

If you want to try a different JVM just place it as the first argument to the startJVM call.

jpype.startJVM('/QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/j9vm/libjvm.so')

But here is the important part. If the JVM has shared libraries that it depends on, then those need to be in the PATH or LIBPATH. Unfortunately, as I can't see your system I can't tell what libraries are required. On some machines this will be $JAVA_HOME/bin, on others it may be $JAVA_HOME/lib. The settings depend on the platform, and as I do not have access to any AIX systems there is no way for me to know.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622592006, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6XC2NHMBYHTFWTI4JTRPND3JANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

I was able to get beyond the jvm error by adding few more in LIBPATH and below is the trace from the latest run, also it generated the following files:- jitdump.20200502.112859.152121.0004.dmp javacore.20200502.112859.152121.0002.txt core.20200502.112859.152121.0001.dmp Snap.20200502.112859.152121.0003.trc

Devender2806 commented 4 years ago

trace_after_jvm.txt

Devender2806 commented 4 years ago

attached are the dmp and trc files after the run javacore.20200502.114020.152159.0002.txt

Thrameos commented 4 years ago

We are definitely getting much closer. We have loaded the hooks but something is failing in the createJVM stage. I am going to modify the ibm branch with more diagnostics, so that we can get a better view. The error that you sent still indicates that is a problem in the JVM itself rather than at JPype. Perhaps one of the shared libraries that is being loaded is wrong.

The other possible problem is the Python library may be compiled with some option that is fundamentally incompatible with the JVM.

The revised version has been pushed to ibm branch of the repo. Please pull a fresh copy and try again.

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

I tried setting up a AIX simulator under qemu. Unfortunately, it is too stripped down and I don't have enough development tools to get the JVM or Python working.

Devender2806 commented 4 years ago

This code on the rep still has issues native/common/jp_platform.cpp: It still throws errors we fixed yesterday. I used the fixed code from my local and the results are same, it still creates the dmp and trc files

On Sat, May 2, 2020 at 12:17 PM Karl Nelson notifications@github.com wrote:

We are definitely getting much closer. We have loaded the hooks but something is failing in the createJVM stage. I am going to modify the ibm branch with more diagnostics, so that we can get a better view. The error that you sent still indicates that is a problem in the JVM itself rather than at JPype. Perhaps one of the shared libraries that is being loaded is wrong.

The other possible problem is the Python library may be compiled with some option that is fundamentally incompatible with the JVM.

The revised version has been pushed to ibm branch of the repo. Please pull a fresh copy and try again.

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

I tried setting up a AIX simulator under qemu. Unfortunately, it is too stripped down and I don't have enough development tools to get the JVM or Python working.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-622986010, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6QWX4XLOWJTIDJXQVTRPRISVANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

i see multiple instance of this on the I server, are these jre instances ? /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/jvmStartPase

Thrameos commented 4 years ago

Can you put in a PR for the IBM branch with all the changes that are necessary?

Thrameos commented 4 years ago

There may be multiple jvms on your system. Please consult with your system administrator about which one is appropriate to use. I can't really offer much advice on that front.

Devender2806 commented 4 years ago

This is the change not updated in the IBM branch and i verified the IBM site we have the correct JAVA_HOME setup and we are using the correct libjvm.so file in the j9vm directory, there are other multiple locations where this file is available but IBM recommends using the one in j9vm.

jweak JPJavaFrame::NewWeakGlobalRef(jobject obj) { JP_TRACE_JAVA("New weak", obj); jweak obj2 = m_Env->NewWeakGlobalRef(obj); JP_TRACE_JAVA("Weak", obj2); return obj2; }

On Sat, May 2, 2020 at 6:49 PM Karl Nelson notifications@github.com wrote:

Can you put in a PR for the IBM branch with all the changes that are necessary?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623030601, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6T46QZAUTXRUC6IY3DRPSWSPANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

The latest run crashed with this:- <?xml version="1.0" standalone="no"?>

#INFO: Crashed in application thread 000000003000E700. #INFO: Found 0 JITed methods on Java stack. #INFO: DUMP FAILED: no methods to recompile

will having a separate JVM for python help ?, could access issues be causing the crash ?

Devender2806 commented 4 years ago

Below is the error when i execute the jvmstart:-

Unhandled exception Type=Illegal instruction vmState=0x00000000 J9Generic_Signal_Number=00000048 Signal_Number=00000004 Error_Value=00000000 Signal_Code=00000033 Handler1=09001000A0534170 Handler2=09001000A0509B20 R0=00000001809C9280 R1=0FFFFFFFFFFFA320 R2=09001000A0537B60 R3=000000000000000E R4=0000000000000018 R5=0FFFFFFFFFFFA450 R6=0000000000000000 R7=000000003000E5D0 R8=0FFFFFFFFFFF9A70 R9=0FFFFFFFFFFF9A60 R10=0000000191979430 R11=0000000000000003 R12=0000000044428842 R13=000000018000F980 R14=000000003000E5A0 R15=000000003000E700 R16=0000000191283858 R17=FFFFFFFFFFFFFFFF R18=00000001802D9AF8 R19=000000003000E700 R20=00000001807645A8 R21=00000001807645A8 R22=0FFFFFFFFFFFA600 R23=000000003003C490 R24=0900000000AB6F7A R25=0000000000000008 R26=0000000000000000 R27=09001000A0537B60 R28=0FFFFFFFFFFFA600 R29=00000001801E1CA0 R30=0900000000940DA4 R31=09001000A04EFFD8 IAR=00000001809C9280 LR=0900000001AED26C MSR=800000001280F032 CTR=00000001809C9280 CR=8242884818000004 FPSCR=8202000000000000 XER=1800000482020000 FPR0 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR1 c3e0000000000000 (f: 0.000000, d: -9.223372e+18) FPR2 41cdcd6500000000 (f: 0.000000, d: 1.000000e+09) FPR3 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR4 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR5 c3e0000000000000 (f: 0.000000, d: -9.223372e+18) FPR6 3fd5555555555611 (f: 1431655936.000000, d: 3.333333e-01) FPR7 412e848000000000 (f: 0.000000, d: 1.000000e+06) FPR8 4010000000000000 (f: 0.000000, d: 4.000000e+00) FPR9 4530000000000000 (f: 0.000000, d: 1.934281e+25) FPR10 412e848000000000 (f: 0.000000, d: 1.000000e+06) FPR11 43300000000f4240 (f: 1000000.000000, d: 4.503600e+15) FPR12 4530000000000000 (f: 0.000000, d: 1.934281e+25) FPR13 0000000000000003 (f: 3.000000, d: 1.482197e-323) FPR14 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR15 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR16 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR17 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR18 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR19 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR20 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR21 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR22 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR23 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR24 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR25 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR26 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR27 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR28 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR29 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR30 0000000000000000 (f: 0.000000, d: 0.000000e+00) FPR31 0000000000000000 (f: 0.000000, d: 0.000000e+00)

Compiled_method=sun/reflect/Reflection.getCallerClass()Ljava/lang/Class; Target=2_90_20191106_432135 (OS/400 V7R4M0) CPU=ppc64 (16 logical CPUs) (0x4000000000 RAM) ----------- Stack Backtrace ----------- sendClinit+0x1f0 (0x0900000000937174 [libj9vm29.so+0x65174]) initializeImpl+0x3fc (0x0900000000933500 [libj9vm29.so+0x61500]) classInitStateMachine__FP10J9VMThreadP7J9Class16J9ClassInitState+0xd88 (0x090000000093460C [libj9vm29.so+0x6260c]) resolveStaticMethodRefInto+0x200 (0x090000000094DB44 [libj9vm29.so+0x7bb44]) resolveStaticMethodRef+0x30 (0x090000000094E094 [libj9vm29.so+0x7c094]) initializeKnownClasses+0x704 (0x090000000355CBA8 [libjclse29.so+0x3dba8]) standardInit+0x674 (0x090000000355EA58 [libjclse29.so+0x3fa58]) J9VMDllMain+0x314 (0x09000000035ABDD8 [libjclse29.so+0x8cdd8]) runJ9VMDllMain@AF74_49+0xf4 (0x09000000008E6FD8 [libj9vm29.so+0x14fd8]) pool_do+0xf8 (0x09000000008F1A5C [libj9vm29.so+0x1fa5c]) IPRA.$runInitializationStage+0xc4 (0x09000000008E4FA8 [libj9vm29.so+0x12fa8]) protectedInitializeJavaVM+0xddc (0x09000000008DCEE0 [libj9vm29.so+0xaee0]) omrsig_protect+0x488 (0x0900000000BE00CC [libj9prt29.so+0x680cc]) initializeJavaVM+0x1e0 (0x09000000008DBC44 [libj9vm29.so+0x9c44]) J9_CreateJavaVM+0xa4 (0x09000000008FB2A8 [libj9vm29.so+0x292a8]) JNI_CreateJavaVM+0xa28 (0x0900000000898C4C [libjvm.so+0x21c4c]) JNI_CreateJavaVM+0x14c (0x0900000000806C90 [libjvm.so+0x2fc90]) J9_i5OS_JNI_CreateJavaVM+0x120 (0x09000000007A5664 [libi5osenv.so+0x2f664]) (0x00000000000036F0 [python+0x36f0])

Thrameos commented 4 years ago

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at... https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

Devender2806 commented 4 years ago

do we know which shared library is causing the issue ?

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at... https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at... https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Commentmailto:comment@noreply.github.com Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at... https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA .

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/jpype-project/jpype/issues/719#issuecomment-623608206, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA.

Devender2806 commented 4 years ago

I will try to work whatever i can from the IBM docs.

On Mon, May 4, 2020 at 12:52 PM Karl Nelson notifications@github.com wrote:

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Comment<mailto: comment@noreply.github.com> Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at...

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/jpype-project/jpype/issues/719#issuecomment-623608206>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623610544, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SUGACLCORRCLMKD6DRP36FZANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

for some reason the threads still refer to /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/compressedrefs even though we are using /QOpenSys/QIBM/ProdData/JavaVM/jdk80/64bit/jre/lib/ppc64/classic

On Mon, May 4, 2020 at 12:52 PM Karl Nelson notifications@github.com wrote:

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Comment<mailto: comment@noreply.github.com> Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at...

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/jpype-project/jpype/issues/719#issuecomment-623608206>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623610544, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SUGACLCORRCLMKD6DRP36FZANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

I attempted a C program to start JVM (libjvm.so) and i see the following message:- ld: 0711-999 SEVERE ERROR: Unexpected system call error. ld:open() Device busy

On Mon, May 4, 2020 at 12:52 PM Karl Nelson notifications@github.com wrote:

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Comment<mailto: comment@noreply.github.com> Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at...

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/jpype-project/jpype/issues/719#issuecomment-623608206>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623610544, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SUGACLCORRCLMKD6DRP36FZANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

I was able to come up with a c code to start JVM based on the documentation, but it still gives the same errors while starting JVM from jpype.

On Mon, May 4, 2020 at 12:52 PM Karl Nelson notifications@github.com wrote:

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Comment<mailto: comment@noreply.github.com> Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at...

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/jpype-project/jpype/issues/719#issuecomment-623608206>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623610544, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SUGACLCORRCLMKD6DRP36FZANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago
#INFO: Crashed in application thread 000000003000E700. #INFO: Found 0 JITed methods on Java stack. #INFO: DUMP FAILED: no methods to recompile

On Mon, May 4, 2020 at 12:52 PM Karl Nelson notifications@github.com wrote:

It is opening the JVM and then dying internally. Thus the best solution it to write a short C or C++ code the replicates the actions of JPype loading and starting the JVM. Once we can isolate the problem as to is it something that JPype (or the python options used to compile JPype) or is it something wrong with the JVM or the libraries/paths used to start it, then we can make some progress.

--Karl

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows 10

From: Devender2806mailto:notifications@github.com Sent: Monday, May 4, 2020 10:47 AM To: jpype-project/jpypemailto:jpype@noreply.github.com Cc: Nelson, Karl E.mailto:nelson85@llnl.gov; Comment<mailto: comment@noreply.github.com> Subject: Re: [jpype-project/jpype] Unable to use JPype on IBM I series (#719)

If i open the latest core.dmp file i see the following: Does this mean it started a JVM ? as earlier the message was always no JRE

PID: 13587 : JRE 1.8.0 OS/400 ppc64-64

and a further info proc revealse:- JIT was enabled for this runtime AOT enabled, FSD disabled, HCR enabled, JIT enabled

On Sun, May 3, 2020 at 7:09 PM Karl Nelson notifications@github.com wrote:

As near as I can tell the issue is outside of JPype as this point. The next step would be copy the code for setting of the JVM (load the library, call createJVM) in a short C++ code. Then repeat it without the dlopen but with a hard linkage. These should be straight out the the IBM documentation.

If it works when calling as a test case but is failing in JPype we can diagnosis the issue by looking for a working pattern. If it fails outside of JPype using the standard IBM recommended createJVM example then the problem with a site problem with the JVM and you will likely need to contact an administrator for help.

An example to start from can be found at...

https://www.ibm.com/support/knowledgecenter/ssw_ibm_i_71/rzaha/invapiex.htm

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/jpype-project/jpype/issues/719#issuecomment-623206895>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/APMFM6UHJ7KNR46ZWDIINUTRPYBR5ANCNFSM4MTKBRIA

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub< https://github.com/jpype-project/jpype/issues/719#issuecomment-623608206>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AG5TEUK5YOPQQAAVOH6JOB3RP35UNANCNFSM4MTKBRIA

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623610544, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6SUGACLCORRCLMKD6DRP36FZANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

Good that means the problem is something in the library path or JVM installation rather than in the Python side.

Devender2806 commented 4 years ago

are there any diagnostics we can do to pin point any library issues or jvm itself ? so that i can take it up with admin

On Mon, May 4, 2020 at 6:18 PM Karl Nelson notifications@github.com wrote:

Good that means the problem is something in the library path or JVM installation rather than in the Python side.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623757812, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6UIJYM7APO7DTTHTKLRP5ENDANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

Unfortunately I have no insight into aix. I usually look on the internet to see if I can find posts on how to find out what shared libraries a shared library depends on.

Devender2806 commented 4 years ago

Have shared the core and trace files with AIX admin and requested him to share with IBM for their review, need to wait for IBM to respond.

On Mon, May 4, 2020 at 6:45 PM Karl Nelson notifications@github.com wrote:

Unfortunately I have no insight into aix. I usually look on the internet to see if I can find posts on how to find out what shared libraries a shared library depends on.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623765660, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6RWXGXKKBIUTNXXNM3RP5HSBANCNFSM4MTKBRIA .

Devender2806 commented 4 years ago

quick question could the dlopen of the libjvm.so file could be a potential reason ?, can we test with a file link ?, is there a way i can do screen share with you to show you the file system ?

On Mon, May 4, 2020 at 6:45 PM Karl Nelson notifications@github.com wrote:

Unfortunately I have no insight into aix. I usually look on the internet to see if I can find posts on how to find out what shared libraries a shared library depends on.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-623765660, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6RWXGXKKBIUTNXXNM3RP5HSBANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

It could. You can try to connect to the JVM using a simple demo program. I know that the diagnostic information on a linked library is much better than the dlopen method. Perhaps it will give you some insight into what is going wrong in the library loading process.

Devender2806 commented 4 years ago

thanks. Another question i have is with the CLASSPATH, it should point to the jdbc jar file correct ? i have ojdbc14.jar in JAVA_HOME/lib, is this the correct file?

On Sat, May 9, 2020 at 7:10 PM Karl Nelson notifications@github.com wrote:

It could. You can try to connect to the JVM using a simple demo program. I know that the diagnostic information on a linked library is much better than the dlopen method. Perhaps it will give you some insight into what is going wrong in the library loading process.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-626252585, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6UXGFP4LQU5Z2J6YE3RQXWHPANCNFSM4MTKBRIA .

Thrameos commented 4 years ago

Likely. Though if it has a native component then you will need to also set the library path. I know that I have a similar issue when using MSSQL. Some jars also have a DLL or SO file that needs to be located. If you get failed to load then it could be an issue.

Thrameos commented 4 years ago

I am going to try to set up the same script for Unix that I did in #729. Perhaps that would help find the issue.

Thrameos commented 4 years ago

I added a similar debugging procedure in #739. Perhaps that would assist you in identifying the issue.

Devender2806 commented 4 years ago

ok i will check. Meanwhile will followup with my admin on IBM's response to the dump and trc files i sent. I suspect the dlopen to be the culprit, as my admin too was talking about creating links for the JVM files during his initial discussion with IBM

On Sat, May 9, 2020 at 8:38 PM Karl Nelson notifications@github.com wrote:

I added a similar debugging procedure in #739 https://github.com/jpype-project/jpype/pull/739. Perhaps that would assist you in identifying the issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jpype-project/jpype/issues/719#issuecomment-626259375, or unsubscribe https://github.com/notifications/unsubscribe-auth/APMFM6QIQYCWXKISEE5DCXTRQYAQ7ANCNFSM4MTKBRIA .