Problem with source build when python is not in regular path

[asrenzo]

Hi, Trying to build python extension from custom python install fails with :

./configure python --config=/some/path/to/python/bin/python-config configuring Python module checking for Python ... not found

./configure: error: no Python found

Sounds like nxt_feature_test of auto/modules/python is not able to compile.

Any idea ?

Regards

[igorsysoev]

On 27 Mar 2018, at 16:03, asrenzo notifications@github.com wrote:

Hi, Trying to build python extension from custom python install fails with :

./configure python --config=/some/path/to/python/bin/python-config configuring Python module checking for Python ... not found

./configure: error: no Python found

Sounds like nxt_feature_test of auto/modules/python is not able to compile.

Any idea ?

What does "tail -20 build/autoconf.err" show?

-- Igor Sysoev http://nginx.com

[asrenzo]

Here it is :

/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(funcobject.o): relocation R_X86_64_32S against symbol `_Py_NoneStruct' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(memoryobject.o): relocation R_X86_64_32 against symbol `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(cframeobject.o): relocation R_X86_64_32 against symbol `PyTuple_Type' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(slp_transfer.o): relocation R_X86_64_32S against symbol `_Py_NoneStruct' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(channelobject.o): relocation R_X86_64_32 against symbol `_Py_NoneStruct' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(getcompiler.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /home/some/path/to/lang/stack279/lib/python2.7/config/libpython2.7.a(structmember.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
----------

        #include <Python.h>

        int main() {
            Py_Initialize();
            return 0;
        }
----------
cc -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -I/home/some/path/to/lang/stack279/include/python2.7 -I/home/some/path/to/lang/stack279/include/python2.7 -o build/autotest build/autotest.c -L/home/some/path/to/lang/stack279/lib/python2.7/config -lpython2.7 -lpthread -ldl -lutil -lm -Xlinker -export-dynamic

[VBart]

@asrenzo It looks like your Python installation doesn't have dynamic library. How did you install it?

[asrenzo]

I build it from source. Standard way. ./configure --prefix=/home/some/path/to/lang/stack279

And I have libs and includes

ls /home/some/path/to/lang/stack279/include/python2.7

abstract.h bytesobject.h complexobject.h errcode.h grammar.h marshal.h object.h pickling pyerrors.h pyport.h rangeobject.h structseq.h unicodeobject.h asdl.h cellobject.h core eval.h import.h memoryobject.h objimpl.h platf pyexpat.h pystate.h setobject.h symtable.h warnings.h ast.h ceval.h cStringIO.h fileobject.h intobject.h metagrammar.h opcode.h pyarena.h pyfpe.h pystrcmp.h sliceobject.h sysmodule.h weakrefobject.h bitset.h classobject.h datetime.h floatobject.h intrcheck.h methodobject.h osdefs.h pycapsule.h pygetopt.h pystrtod.h stackless_api.h timefuncs.h boolobject.h cobject.h descrobject.h frameobject.h iterobject.h modsupport.h parsetok.h pyconfig.h pymacconfig.h Python-ast.h stackless.h token.h bufferobject.h codecs.h dictobject.h funcobject.h listobject.h module patchlevel.h pyctype.h pymactoolbox.h Python.h stackless_version.h traceback.h bytearrayobject.h code.h dtoa.h genobject.h longintrepr.h moduleobject.h pgen.h py_curses.h pymath.h pythonrun.h stringobject.h tupleobject.h bytes_methods.h compile.h enumobject.h graminit.h longobject.h node.h pgenheaders.h pydebug.h pymem.h pythread.h structmember.h ucnhash.h

And ls -l /home/some/path/to/lang/stack279/lib/python2.7/config

config.c config.c.in install-sh libpython2.7.a Makefile makesetup python.o Setup Setup.config Setup.local

[igorsysoev]

On 27 Mar 2018, at 16:52, asrenzo notifications@github.com wrote:

I build it from source. Standard way. ./configure --prefix=/home/some/path/to/lang/stack279

Try to add --enable-shared

-- Igor Sysoev http://nginx.com

[asrenzo]

OK. Rebuilding my python with --enable-shared sounds better

[igorsysoev]

On 27 Mar 2018, at 17:10, asrenzo notifications@github.com wrote:

OK. Rebuilding my python with --enable-shared sounds better

Probably you need also to add --lib-path to unit configure:

./configure python --config=/some/path/to/python/bin/python-config --lib-path=/home/some/path/to/lang/stack279/lib/

if "python-config --ldflags" does not return it.

-- Igor Sysoev http://nginx.com

[nshadrin]

@asrenzo Does this help? Can we close this issue?

[samthesuperhero]

Hi all, have the same issue,

sudo ./configure python --module=python37 --config=/home/centos/py37.source/Python-3.7.0/python-config --lib-path=/usr/local/lib

and the output: configuring Python module checking for Python ... not found

./configure: error: no Python found.

And,

tail -20 build/autoconf.err

configuring Python module ... /home/centos/py37.source

checking for Python build/autotest.c:2:28: fatal error: Python.h: No such file or directory

include

                        ^

compilation terminated.

    #include <Python.h>

    int main() {
        Py_Initialize();
        return 0;
    }

---

cc -pipe -fPIC -fvisibility=hidden -O -W -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-prototypes -Werror -g -I/home/centos/py37.source/include/python3.7m -I/home/centos/py37.source/include/python3.7m -o build/autotest build/autotest.c -L/home/centos/py37.source/lib -lpython3.7m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -L/usr/local/lib -Wl,-rpath /usr/local/lib

Can it be anyhow solved ??

[VBart]

@samthesuperhero It looks like a problem with your python build. Could you show the output of /home/centos/py37.source/Python-3.7.0/python-config --includes and /home/centos/py37.source/Python-3.7.0/python-config --ldflags?

[samthesuperhero]

Hi , thx for fast answer, in fact I just managed this particular problem by just using python-config from /usr/local/bin,

BUT - after, i met the new issue:

1. it doesn't see version, [centos@pavel-fomin-82-i1 unit]$ sudo ./configure python --module=python37 --config=/usr/local/bin/python3.7-config configuring Python module checking for Python ... found checking for Python version ... not found

   • Python module: python37.unit.so

2. It requested to user prefix, [centos@pavel-fomin-82-i1 unit]$ sudo make install

error: to make install you need either to configure --prefix option or to set DESTDIR environment variable.

make: *** [install-check] Error 1

3. And - after I used /usr/local as prefix, it now doesn't see python module, I tried a number of possible names of "type" parameter.... [centos@pavel-fomin-82-i1 ~]$ sudo curl -X PUT -d @/home/centos/unit-app-uploadapp.json --unix-socket /usr/local/control.unit.sock 'http://localhost/config/applications/uploadapp2' { "error": "Invalid configuration.", "detail": "The module to run \"python37\" is not found among the available application modules." }

with conf JSON: { "type": "python37", "processes": 4, "user": "root", "group": "root", "path": "/home/centos", "module": "uploadapp1" }

[VBart]

1. Please, check autoconf.err in order to investigate why no version has been shown on configure stage.
2. In order to use make install you have to specify all the paths (or at least an install prefix).
3. The module file name doesn't matter. The type should be "python", or "python 3.7".

[samthesuperhero]

1. The record is: configuring Python module ... /usr/local

   checking for Python

   checking for Python version build/autotest: error while loading shared libraries: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

what's wrong ?

2. OK I see

3. both options don't work! (btw, how this "type" is related to module name in modules/ ??)

[VBart]

That means dynamic loader is unable to find your Python shared library, because the library isn't in the standard paths. In this case you have to specify a path to library directory with the --lib-path= configure option (that directory should contain libpython3.7m.so.1.0).

Please check unit.log. It probably contains the same error, that loader unable to find Python library.

The type field isn't related to module names at all. Module name can be arbitrary. It's needed only to separate make targets and file names during building process.

[samthesuperhero]

Great, solved!! +1 question: now compared to package distribution with python2.7 - it stopped to create .pyc files from my wsgi scripts, why so? (before after I created application, it created the corresponding .pyc code)

[VBart]

@samthesuperhero Starting from Python 3.2 they are stored in special __pycache__ directories. See PEP 3147 for details.