import fcntl error: undefined symbol: PyExc_ValueError

st4rfall commented 3 years ago

Describe the problem I have to use Java to drive my tensorflow parogram but meet error when trying to import fcntl. So I make a demo and package it a jar file(jep is packaged too). After executing java -jar I get this error:

Exception in thread "main" jep.JepException: <class 'ImportError'>: /root/python379tgz/Python-3.7.9-compiled/lib/python3.7/lib-dynload/fcntl.cpy
thon-37m-x86_64-linux-gnu.so: undefined symbol: PyExc_ValueError
        at <string>.<module>(<string>:1)
        at jep.Jep.exec(Native Method)
        at jep.Jep.exec(Jep.java:478)
        at com.xhunter.pack.App.main(App.java:24)

here is the code

import jep.*;

public class App {
    public static void main( String[] args ) throws Exception {
        PyConfig pyConfig = new PyConfig();
        pyConfig.setPythonHome("/root/python379tgz/Python-3.7.9-compiled");
        MainInterpreter.setJepLibraryPath("/root/python379tgz/Python-3.7.9-compiled/lib/python3.7/site-packages/jep/libjep.so");
        MainInterpreter.setInitParams(pyConfig);
        Interpreter interp = new SharedInterpreter();
        interp.exec("print('print success')");
        interp.exec("import sys");
        interp.exec("import fcntl");
    }
}

Environment (please complete the following information):

OS Platform, Distribution, and Version: CentOS Linux release 7.5.1804 (Core)
Python Distribution and Version: python3.7
Java Distribution and Version: openjdk version "1.8.0_292" OpenJDK Runtime Environment (build 1.8.0_292-b10) OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)
Jep Version: 3.9.1
Python packages used (e.g. numpy, pandas, tensorflow):

Logs Please attach the complete console output of the build.

Additional context I compiled my python the folloing steps

./configure --prefix=/root/python379tgz/Python-3.7.9-compiled/ -CFLAGS='-fPIC'
make
gcc -pthread  -Wl,-rpath=.   -Xlinker -export-dynamic -o python Programs/python.o -L. -lpython3.7m -lcrypt -lpthread -ldl  -lutil   -lm
make install

and my python works fine importing fcntl:

[root@localhost python379tgz]# /root/python379tgz/Python-3.7.9-compiled/bin/python3.7
Python 3.7.9 (default, Jul 30 2021, 12:33:12)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fcntl
>>> import tensorflow

any ideas?

st4rfall commented 3 years ago

change src code and rebuild to load the library https://github.com/ninia/jep/blob/8a2746a2ff67d219d76733f05d526e4c65e9a177/src/main/c/Jep/pyembed.c#L375

void* dlresult = dlopen("/root/python379tgz/Python-3.7.9-compile-enableshared/lib/libpython3.7m.so.1.0", RTLD_LAZY | RTLD_NOLOAD | RTLD_GLOBAL);

and load the library in my java app too

System.load("/root/python379tgz/Python-3.7.9-compile-enableshared/lib/libpython3.7m.so.1.0");

problem solved. very inelegant way

ndjensen commented 3 years ago

Was the directory containing libpython in your LD_LIBRARY_PATH environment variable?

ninia / jep

import fcntl error: undefined symbol: PyExc_ValueError #342