Open stitch opened 2 years ago
wcawijngaards
commented
2 years ago The python detection code uses the variable PYTHON_VERSION that can be passed to detect python directories. Try setting PYTHON_VERSION=3.8 with like ./configure --with-pyunbound PYTHON_VERSION=3.8
The configure script then uses that version of python distutils with python3.8 and gets the directories needed.
wcawijngaards
commented
2 years ago If that does not work you can influence the results that are gathered from the python3.8 version, by setting the variables: PYTHON (to set the python commandline executable to use), and the directories used for compilation individually with PYTHON_CPPFLAGS and PYTHON_LDFLAGS and PYTHON_SITE_PKG to the directories that you want to use. But the easy method is to set PYTHON_VERSION and have it detect the settings from there.
stitch
commented
2 years ago Thanks for the quick response!
I see, switching the python version around does seem to discover alternative folders. Those seem to be correct-ish, while none of them contains Python.h. Setting PYTHON_LDFLAGS to a directory that does contain these file still does not has any impact on the outcome.
Switching around different python versions results in paths that seem fine. But still the compilation fails.
3.7:
checking for python3.7... /usr/local/bin/python3.7
checking for the distutils Python package... yes
checking for Python include path... -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m
checking for Python library path... -L/Library/Frameworks/Python.framework/Versions/3.7/lib -L/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7
checking for Python site-packages path... /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages3.8:
checking for python3.8... /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
checking for the distutils Python package... yes
checking for Python include path... -I/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8
checking for Python library path... -L/Library/Frameworks/Python.framework/Versions/3.8/lib -L/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8
checking for Python site-packages path... /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages3.9:
checking for python3.9... /opt/homebrew/bin/python3.9
checking for the distutils Python package... yes
checking for Python include path... -I/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
checking for Python library path... -L/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib -L/opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9
checking for Python site-packages path... /opt/homebrew/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packagesCatting conftest.err results in:
undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-dcb158.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
noSo the problem is not with the paths, but with the x86 / arm64 issue. I'm trying to compile it on an arm64 macbook (m1). Perhaps that usecase is not supported?
stitch
commented
2 years ago Seems to be a deadlock:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis
results in: ld: symbol(s) not found for architecture arm64
arch -x86_64 ./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis
results in ld: symbol(s) not found for architecture x86_64
wcawijngaards
commented
2 years ago Unbound and I guess python can compile for arm64 just fine, we have unit tests that compile on arm64. Perhaps the homebrew installed binaries are for the wrong architecture, x86 instead of arm64?
The library path that it has found contains the python library? And it tries to link that in the config.log output? If so I guess you are right that it is the arm64.
What is it exactly trying to do that gives the symbol not found error? config.log says what it does.
stitch
commented
2 years ago I installed python@3.8 from brew and followed the instructions. The paths are:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis PYTHON_LDFLAGS=/opt/homebrew/opt/python@3.8/lib PYTHON_CPPFLAGS=/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8
Config.log says: ` conftest.c:85:26: fatal error: 'Python.h' file not found
^~~~~~~~~~1 error generated. configure:17396: $? = 1 configure: failed program was: `
The contents of ls /opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 includes Python.h and many other .h files... specifically for version 3.8 of python as they come with the brew package. I've also tried several completely new other python installs.
The permissions of Python.h:
-rw-r--r-- 1 someuser somegroup 3615 Aug 30 18:42 Python.h
Gcc has root permissions.
The content of Python.h looks fine.
Running the compliation command as root still results in Python.h file not found.
The gcc version i'm using:
(.venv) macbookprom1:unbound stitch$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 13.0.0 (clang-1300.0.29.3) Target: arm64-apple-darwin20.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
I've tried it with the homebrew version of gcc:
/opt/homebrew/Cellar/gcc/11.2.0_1/bin/gcc-11 -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread /opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 conftest.c /opt/homebrew/opt/python@3.8/lib conftest.c:1:10: fatal error: Python.h: No such file or directory 1 | #include <Python.h>
Here is the more verbose config.log output:
configure:17251: checking for python
configure:17269: found /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/bin/python
configure:17281: result: /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/bin/python
configure:17302: checking for the distutils Python package
configure:17305: result: yes
configure:17319: checking for Python include path
configure:17329: result: /opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8
configure:17336: checking for Python library path
configure:17342: result: /opt/homebrew/opt/python@3.8/lib
configure:17354: checking for Python site-packages path
configure:17360: result: /Users/stitch/Documents/_webdevelopment/internetnl/Internet.nl/.venv/lib/python3.8/site-packages
configure:17367: checking consistency of all components of python development environment
configure:17396: gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread /opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 conftest.c /opt/homebrew/opt/python@3.8/lib >&5
conftest.c:85:26: fatal error: 'Python.h' file not found
#include <Python.h>
^~~~~~~~~~
1 error generated.
configure:17396: $? = 1My compiler skills are lacking clearly. All my assumptions of including files based on a path are now over board. Where am i going wrong?
stitch
commented
2 years ago For reference:
ls /opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8
Python-ast.h cellobject.h dictobject.h genobject.h marshal.h osdefs.h pydtrace.h pyport.h structmember.h warnings.h
Python.h ceval.h dtoa.h graminit.h memoryobject.h osmodule.h pydtrace_probes.h pystate.h structseq.h weakrefobject.h
abstract.h classobject.h dynamic_annotations.h grammar.h methodobject.h parsetok.h pyerrors.h pystrcmp.h symtable.h
asdl.h code.h enumobject.h import.h modsupport.h patchlevel.h pyexpat.h pystrhex.h sysmodule.h
ast.h codecs.h errcode.h internal moduleobject.h picklebufobject.h pyfpe.h pystrtod.h token.h
bitset.h compile.h eval.h interpreteridobject.h namespaceobject.h py_curses.h pyhash.h pythonrun.h traceback.h
bltinmodule.h complexobject.h fileobject.h intrcheck.h node.h pyarena.h pylifecycle.h pythread.h tracemalloc.h
boolobject.h context.h fileutils.h iterobject.h object.h pycapsule.h pymacconfig.h pytime.h tupleobject.h
bytearrayobject.h cpython floatobject.h listobject.h objimpl.h pyconfig.h pymacro.h rangeobject.h typeslots.h
bytes_methods.h datetime.h frameobject.h longintrepr.h odictobject.h pyctype.h pymath.h setobject.h ucnhash.h
bytesobject.h descrobject.h funcobject.h longobject.h opcode.h pydebug.h pymem.h sliceobject.h unicodeobject.h
You are missing the -L and -I flags that the LDFLAGS and CPPFLAGS need in front of the directory. That is needed to specify how the compile treats the directory, for searching libraries or searching includes.
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis PYTHON_LDFLAGS=-L/opt/homebrew/opt/python@3.8/lib PYTHON_CPPFLAGS=-I/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8
stitch
commented
2 years ago Thanks for the quick response! :)
Ah, that's how it's picked up. Still being squished between non existing symbols.
arm64 (default):
gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread -I/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 manualtest.c -L/opt/homebrew/opt/python@3.8/lib
undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-6e745a.o
ld: symbol(s) not found for architecture arm64x86_64:
arch -x86_64 gcc -o conftest -g -O2 -flto -D_THREAD_SAFE -pthread -I/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8 manualtest.c -L/opt/homebrew/opt/python@3.8/lib
undef: _Py_Initialize
Undefined symbols for architecture x86_64:
"_Py_Initialize", referenced from:
_main in cc-cccdce.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)The python from brew is a 64 bit one:
file /opt/homebrew/opt/python@3.8/bin/python3
/opt/homebrew/opt/python@3.8/bin/python3: Mach-O 64-bit executable arm64That could be because -lpython3.8 is missing from the commandline, because you override the python checks with your own values. Try adding -lpython3.8 to the PYTHON_LDFLAGS:
./configure --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhredis "PYTHON_LDFLAGS=-L/opt/homebrew/opt/python@3.8/lib -lpython3.8" PYTHON_CPPFLAGS=-I/opt/homebrew/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/include/python3.8
stitch
commented
2 years ago Should 'lpython3.8' be the python binary?
In that case, no luck pointing to it directly and indirectly.
ld: library not found for -lpython3.8
wcawijngaards
commented
2 years ago Yes it is the python binary, the libpython file. Without the 'lib' part, or the '.so' or '.a' file extension. The file should be in the lib directory.
stitch
commented
2 years ago What's the content of a normal 'lib' directory?
The instructions from brew point to this:
ls /opt/homebrew/opt/python@3.8/lib
pkgconfigDoesn't seem right :)
wcawijngaards
commented
2 years ago You need a file like libpython3.8.a or libpython3.8.so or something like that. Pass the directory where that file is, perhaps in lib64, if a directory with that name exists?
stitch
commented
2 years ago Still no luck, it seems like an arm64 version of python is being mixed with the x86_64 one, even after making all python variables explicitly use the x86_64 one. I explicitly need an x86_64 version of the library, given there are some dependencies in the clients repo that will probably not support arm64.
1: get the x86_64 version of brew to install python3.9:
arch -x86_64 /bin/bash
/usr/local/Homebrew/bin/brew install python@3.92: Verify that the this cannot be run on arm64:
arch -arm64 /usr/local/Cellar/python@3.9/3.9.9/bin/python3.9
arch: posix_spawnp: /usr/local/Cellar/python@3.9/3.9.9/bin/python3.9: Bad CPU type in executable
bash-3.2$ arch -x86_64 /usr/local/Cellar/python@3.9/3.9.9/bin/python3.9
Python 3.9.9 (main, Nov 21 2021, 03:23:44)
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.3: Check the contents of the include path (-I):
ls /usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
Python-ast.h cellobject.h descrobject.h frameobject.h iterobject.h node.h py_curses.h pyfpe.h pystrcmp.h structseq.h warnings.h
Python.h ceval.h dictobject.h funcobject.h listobject.h object.h pyarena.h pyframe.h pystrhex.h symtable.h weakrefobject.h
abstract.h classobject.h dynamic_annotations.h genericaliasobject.h longintrepr.h objimpl.h pycapsule.h pyhash.h pystrtod.h sysmodule.h
asdl.h code.h enumobject.h genobject.h longobject.h odictobject.h pyconfig.h pylifecycle.h pythonrun.h token.h
ast.h codecs.h errcode.h graminit.h marshal.h opcode.h pyctype.h pymacconfig.h pythread.h traceback.h
bitset.h compile.h eval.h grammar.h memoryobject.h osdefs.h pydebug.h pymacro.h pytime.h tracemalloc.h
bltinmodule.h complexobject.h exports.h import.h methodobject.h osmodule.h pydtrace.h pymath.h rangeobject.h tupleobject.h
boolobject.h context.h fileobject.h internal modsupport.h parsetok.h pydtrace_probes.h pymem.h setobject.h typeslots.h
bytearrayobject.h cpython fileutils.h interpreteridobject.h moduleobject.h patchlevel.h pyerrors.h pyport.h sliceobject.h ucnhash.h
bytesobject.h datetime.h floatobject.h intrcheck.h namespaceobject.h picklebufobject.h pyexpat.h pystate.h structmember.h unicodeobject.h4: check the contents of the library paths:
ls /usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib
libpython3.9.dylib pkgconfig python3.9ls /usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin
Makefile Setup Setup.local config.c config.c.in install-sh libpython3.9.a libpython3.9.dylib makesetup python-config.py python.ols /usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9
LICENSE.txt binhex.py difflib.py imp.py pickletools.py site.py timeit.py
__future__.py bisect.py dis.py importlib pipes.py sitecustomize.py tkinter
__phello__.foo.py bz2.py distutils inspect.py pkgutil.py smtpd.py token.py
__pycache__ cProfile.py doctest.py io.py platform.py smtplib.py tokenize.py
_aix_support.py calendar.py email ipaddress.py plistlib.py sndhdr.py trace.py
_bootlocale.py cgi.py encodings json poplib.py socket.py traceback.py
_bootsubprocess.py cgitb.py ensurepip keyword.py posixpath.py socketserver.py tracemalloc.py
_collections_abc.py chunk.py enum.py lib-dynload pprint.py sqlite3 tty.py
_compat_pickle.py cmd.py filecmp.py lib2to3 profile.py sre_compile.py turtle.py
_compression.py code.py fileinput.py linecache.py pstats.py sre_constants.py turtledemo
_markupbase.py codecs.py fnmatch.py locale.py pty.py sre_parse.py types.py
_osx_support.py codeop.py formatter.py logging py_compile.py ssl.py typing.py
_py_abc.py collections fractions.py lzma.py pyclbr.py stat.py unittest
_pydecimal.py colorsys.py ftplib.py mailbox.py pydoc.py statistics.py urllib
_pyio.py compileall.py functools.py mailcap.py pydoc_data string.py uu.py
_sitebuiltins.py concurrent genericpath.py mimetypes.py queue.py stringprep.py uuid.py
_strptime.py config-3.9-darwin getopt.py modulefinder.py quopri.py struct.py venv
_sysconfigdata__darwin_darwin.py configparser.py getpass.py multiprocessing random.py subprocess.py warnings.py
_threading_local.py contextlib.py gettext.py netrc.py re.py sunau.py wave.py
_weakrefset.py contextvars.py glob.py nntplib.py reprlib.py symbol.py weakref.py
abc.py copy.py graphlib.py ntpath.py rlcompleter.py symtable.py webbrowser.py
aifc.py copyreg.py gzip.py nturl2path.py runpy.py sysconfig.py wsgiref
antigravity.py crypt.py hashlib.py numbers.py sched.py tabnanny.py xdrlib.py
argparse.py csv.py heapq.py opcode.py secrets.py tarfile.py xml
ast.py ctypes hmac.py operator.py selectors.py telnetlib.py xmlrpc
asynchat.py curses html optparse.py shelve.py tempfile.py zipapp.py
asyncio dataclasses.py http os.py shlex.py test zipfile.py
asyncore.py datetime.py idlelib pathlib.py shutil.py textwrap.py zipimport.py
base64.py dbm imaplib.py pdb.py signal.py this.py zoneinfo
bdb.py decimal.py imghdr.py pickle.py site-packages threading.py5: Combine all of the above into a command that specifies the interpreter and all required paths into one thing:
cd unbound && /usr/bin/arch -x86_64 ./configure --prefix=/home/$(USER)/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis PYTHON="/usr/local/Cellar/python@3.9/3.9.9/bin/python3.9" PYTHON_SITE_PKG=$(ROOT_DIR)/.venv/lib/python3.9/site-packages PYTHON_LDFLAGS="-L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 -L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin -L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib" PYTHON_CPPFLAGS="-I/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9" && make install6: Add some inspections to configure and see what's really going wrong:
cat conftest.err
cat conftest.$ac_objext
cat conftest$ac_exeext
cat conftest.$ac_ext7: See the output:
checking for python... /usr/local/Cellar/python@3.9/3.9.9/bin/python3.9
checking for the distutils Python package... yes
checking for Python include path... -I/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/include/python3.9
checking for Python library path... -L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9 -L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/config-3.9-darwin -L/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib
checking for Python site-packages path... /Users/stitch/Documents/_webdevelopment/internetnl2/Internet.nl/.venv/lib/python3.9/site-packages
checking consistency of all components of python development environment... undef: _Py_Initialize
Undefined symbols for architecture arm64:
"_Py_Initialize", referenced from:
_main in cc-020cd3.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)8: Possible conclusions:
Any more help would be appreciated :)
edit: paths
stitch
commented
2 years ago On a fresh debian 10 install, after installing python3 from source:
I've found a bug in the configure script: it suppresses any python compilation error messages, and replacing that with an irrelevant message. Above on a mac i got a symbol warning, after the debian machine i get a 'gcc: error: libpython3.9.a: No such file or directory'.
Header files are present. I cannot find the libpython file indeed. And since there is no python3-dev on this apt, and deadsnakes does not provide it for debian 10, and there are no seperate instructions to install python3 development tools separately (i guess they would be installed when you install python3 from source, but no). Google doesnt help here.
Trying again with a fresh debian 11 vm.
wcawijngaards
commented
2 years ago The checking consistency phase should log the results of that in the config.log file. There you should be able to find the compiler command that was executed and the failing results from it. Like file not found errors.
The PYTHON_LDFLAGS also needs to have -lpython in there. This is likely why it does not link. Put that after the directory -L contents. The second time it does not link with libpython3.9.a, I am thinking you did not set PYTHON_LDFLAGS, otherwise I am unsure how it would reference that file.
Also set the variable PYTHON_LIBDIR to the name of one directory, one with python.dylib in there for runtime.
stitch
commented
2 years ago There we are:
.unbound-x86-3.8:
# For m1 users:
# arch -x86_64 /bin/bash
# /usr/local/Homebrew/bin/brew install python@3.8
# brew unlink python@3.8 && brew link python@3.8
# /usr/local/Homebrew/bin/brew install libevent
# /usr/local/Homebrew/bin/brew install hiredis
rm -rf unbound
git clone https://github.com/internetstandards/unbound
cd unbound && /usr/bin/arch -x86_64 ./configure --enable-internetnl --with-pyunbound --with-libevent --with-libhiredis PYTHON="/usr/local/Cellar/python@3.8/3.8.12_1/bin/python3.8" PYTHON_SITE_PKG=$(ROOT_DIR)/.venv/lib/python3.8/site-packages PYTHON_LDFLAGS="-L/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8 -L/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/config-3.8-darwin -L/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib -lpython3.8" PYTHON_CPPFLAGS="-I/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/include/python3.8" PYTHON_LIBDIR="/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib" && make install
touch .unbound-x86-3.8Thanks for the quick and thorough support!
stitch
commented
2 years ago Reopening it because of the same miserable experience on a random debian machine where a python 3.7 is needed instead of 3.9.
I've installed python 3.7 via source and it's missing the library dirs. There is little instruction on how to get it or where to start and searching is not really delvering.
Seriously, the python dependencies are a hurdle every installation. Can you make this process any easier or better? It's just awful.
wcawijngaards
commented
2 years ago Usually the python dependencies are installed from the package management system, and I have not had other complaints on how it could be better. The configure now wants to use the python settings for its directories, or you can set them with flags to configure.
So the python is needed that is not the one from the package management system. So you need to select it somehow with configure flags. Configure outputs python checks, and those also have entries in config.log. Does it make it through that and compile or fail at build stage? What is wrong?
stitch
commented
2 years ago That's weird, i'm wondering how nobody runs into this.
When the python deps are installed from the package manager it all works fine, you get the development headers and whatnot and simply compiling and running works. It goes wrong when trying to install a custom python version, when the software you're working on wants an older version.
So under debian 11, there is no trivial way to get something like "libpython3.7-dev" and so it seems that getting unbound compiled for python3.7 is impossible. Perhaps you know better?
wcawijngaards
commented
2 years ago I have not really experience with the matter. I would assume from other packages that you should install the python3.7 in some other destination and then set PYTHON_VERSION=3.7 on the ./configure command line. This approach has worked for me. Make sure that the python3.7 binary is in the executable lookup path when configure runs.
mjpieters
commented
2 years ago Why not use pythonX.Y-config script to obtain the information? I suspect I know the answer: because unbound's Python integration predates the pythonX.Y-config scripts.
However, using that tool with the right switches works to configure unbound, for any recent X.Y version of Python:
./configure [other options] \
--with-pythonmodule \
PYTHON_VERSION="X.Y" \
PYTHON_LDFLAGS="$(pythonX.Y-config --ldflags --embed)" \
PYTHON_CPPFLAGS="$(pythonX.Y-config --includes)"Note: the X.Y values above should be replaced by your actual Python version.
I used the --embed switch for pythonX.Y-config, it's the crucial part here. It produces the LDFLAGS values for embedding the interpreter, meaning it adds -lpythonX.Y to the flags. This value is built from sysconfig.get_config_var("VERSION") and sys.abiflags (for sys.abiflags you can also use sysconfig.get_config_var("ABIFLAGS")).
The unbound-project supplied acx_python.m4 instead uses sysconfig.get_config_var('BLDLIBRARY'), which is libpythonX.Y.a (so including the lib prefix and .a extension), and the script doesn't then prefix this with -l, and so is doomed to fail.
jeffgogel
commented
7 months ago @mjpieters
Thank you for this!! Using those python-config options allowed me to compile on Ubuntu 22.04 w/ Python 3.10.
./configure --prefix=/usr --includedir=\${prefix}/include --infodir=\${prefix}/share/info --mandir=\${prefix}/share/man --localstatedir=/var --runstatedir=/run --sysconfdir=/etc --with-chroot-dir= --with-dnstap-socket-path=/run/dnstap.sock --with-libevent --with-libhiredis --with-libnghttp2 --with-pidfile=/run/unbound.pid --with-pythonmodule --with-pyunbound --with-rootkey-file=/var/lib/unbound/root.key --disable-dependency-tracking --disable-flto --disable-maintainer-mode --disable-option-checking --disable-rpath --disable-silent-rules --enable-cachedb --enable-dnstap --enable-subnet --enable-systemd --enable-tfo-client --enable-tfo-server PYTHON_VERSION=3.10 PYTHON_LDFLAGS="$(python3.10-config --ldflags --embed)" PYTHON_CPPFLAGS="$(python3.10-config --includes)" CFLAGS="-O2"
I'm trying to compile unbound on a mac, i specifically need the python bindings for internet.nl development. To get these bindings the developer headers from python are needed. I've used a dozen different directories for the header files, from both Mac OS and Homebrew. None of them seem to satisfy the requirements from
configure, even if they contain python.h or the proper dylibs.For example:
./configure LDFLAGS="-L/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers" --prefix=$HOME/usr/local --enable-internetnl --with-pyunbound --with-libevent --with-libhiredisOther pats tried are:
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/opt/homebrew/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/Current/include/python3.8//opt/homebrew/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/include/python3.8/... and any that contain
libExpected behavior Compilation of unbound or any more specific message what is wrong.
System:
unbound -Voutput: -Additional information Add any other information that you may have gathered about the issue here.