bfgroup / b2

B2 makes it easy to build C++ projects, everywhere.
https://www.bfgroup.xyz/b2/
Boost Software License 1.0
81 stars 230 forks source link

Python gets detected correctly but the Header File pyconfig.h can not be found - MSYS2 MinGW-w64 10.2.0 #17

Open MeerMusik opened 3 years ago

MeerMusik commented 3 years ago

Make sure you completed the following tasks

Environment and version details

Bootstrapping is done. To build, run:

./b2

To generate header files, run:

./b2 headers

The configuration generated uses gcc to build by default. If that is unintended either use the --with-toolset option or adjust configuration, by editing 'project-config.jam'.

Further information:

Brief problem description

The Header File 'pyconfig.h' does not get found and compilation errors out:

In file included from ./boost/python/detail/prefix.hpp:13,
                 from ./boost/python/list.hpp:8,
                 from libs\python\src\list.cpp:5:
./boost/python/detail/wrap_python.hpp:57:11: fatal error: pyconfig.h: No such file or directory
   57 | # include <pyconfig.h>

Steps to reproduce the issue

I run the following Script (Snippet) to build the Boost Libraries from a locally fresh cloned Git Repository (Master Branch, 1.76):

if [ ! -d ${SYSTEMDRIVE}/Dev/BoostSrc/ ]; then cp --force --recursive --verbose ${SYSTEMDRIVE}/Dev/GitRepos/boost/ ${SYSTEMDRIVE}/Dev/BoostSrc/
fi
cd ${SYSTEMDRIVE}/Dev/BoostSrc/
./bootstrap.sh --with-icu=${SYSTEMDRIVE}/Dev/ICUx64MinGW/lib/ --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/debug/
./b2 install --debug-configuration release link=static --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/debug/

Actual behavior summary

Also random stuff like this from the Internet did not help:

export C_INCLUDE_PATH="$C_INCLUDE_PATH:C:/Dev/msys64/mingw64/include/python3.8/"
export CPLUS_INCLUDE_PATH="$CPLUS_INCLUDE_PATH:C:/Dev/msys64/mingw64/include/python3.8/"
echo "using python : 3.8 ;" >> ${SYSTEMDRIVE}/Dev/BoostSrc/tools/build/src/site-config.jam

If a project of yours is blocked due to this bug, please, mention it explicitly.

Here is the cut Output from './b2 install -d2 --debug-configuration release link=static --prefix=${SYSTEMDRIVE}/Dev/Sources/GTK4/boost/x64/release/ 1>aaa.txt 2>&1':

notice: found boost-build.jam at C:/Dev/BoostSrc/boost-build.jam
notice: loading B2 from C:/Dev/BoostSrc/tools/build/src/bootstrap.jam
notice: Searching 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.'.
notice: Searching 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.' for user-config configuration file 'user-config.jam'.
notice: Configuration file 'user-config.jam' not found in 'C:\Users\Oliver' 'C:\Dev\msys64\home\Oliver' 'C:\Users\Oliver' 'C:/Dev/BoostSrc/tools/build/src' 'C:/Dev/BoostSrc/tools/build/src/kernel' 'C:/Dev/BoostSrc/tools/build/src/util' 'C:/Dev/BoostSrc/tools/build/src/build' 'C:/Dev/BoostSrc/tools/build/src/tools' 'C:/Dev/BoostSrc/tools/build/src/contrib' 'C:/Dev/BoostSrc/tools/build/src/.'.
notice: Searching '.' for project-config configuration file 'project-config.jam'.
notice: Loading project-config configuration file 'project-config.jam' from '.'.
notice: will use 'g++' for gcc, condition <toolset>gcc-10.2.0
notice: using gcc libraries :: <toolset>gcc-10.2.0 :: C:\Dev\msys64\mingw64/bin C:\Dev\msys64\mingw64/lib C:\Dev\msys64\mingw64/lib32 C:\Dev\msys64\mingw64/lib64
notice: using gcc archiver :: <toolset>gcc-10.2.0 :: C:/Dev/msys64/mingw64/x86_64-w64-mingw32/bin/ar.exe
notice: using gcc ranlib :: <toolset>gcc-10.2.0 :: C:/Dev/msys64/mingw64/x86_64-w64-mingw32/bin/ranlib.exe
notice: using rc compiler :: <toolset>gcc-10.2.0 :: C:\Dev\msys64\mingw64\bin\windres.exe
notice: [python-cfg] Configuring python...
notice: [python-cfg]   user-specified version: "3.8"
notice: [python-cfg]   user-specified cmd-or-prefix: "C:/Dev/msys64/mingw64"
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\PCBuild8\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\PCBuild8\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\PCBuild\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\PCBuild\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "C:/Dev/msys64/mingw64\python"...
notice: [python-cfg] running command 'C:/Dev/msys64/mingw64\python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...does not invoke a working interpreter
notice: [python-cfg] Checking interpreter command "python"...
notice: [python-cfg] running command 'DIR /-C /A:S "C:\Dev\msys64\mingw64\bin\python.exe" 2>&1'
notice: [python-cfg] running command 'python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "python"
notice: [python-cfg]   include path: "C:/Dev/msys64/mingw64\Include"
notice: [python-cfg]   library path: "C:/Dev/msys64/mingw64\libs"
notice: [python-cfg]   DLL search path: "C:/Dev/msys64/mingw64"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command 'python -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy disabled. Reason:
notice: [python-cfg]   python -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())" aborted with 
notice: [python-cfg]   Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'numpy'
notice: [zlib] Using pre-installed library
notice: [zlib] Condition
notice: [bzip2] Using pre-installed library
notice: [bzip2] Condition
notice: [lzma] Using pre-installed library
notice: [lzma] Condition
notice: [zstd] Using pre-installed library
notice: [zstd] Condition
Performing configuration checks

    - default address-model    : 64-bit [1]
    - default architecture     : x86 [1]
    - symlinks supported       : yes
    - compiler supports SSE2   : yes [2]
    - compiler supports SSE4.1 : yes [2]
    - has synchronization.lib  : no [2]
    - C++11 mutex              : yes [2]
    - has BCrypt API           : yes [2]
    - has stat::st_mtim        : no [2]
    - has stat::st_mtimensec   : no [2]
    - has stat::st_mtimespec   : no [2]
    - has stat::st_birthtim    : no [2]
    - has stat::st_birthtimensec : no [2]
    - has stat::st_birthtimespec : no [2]
    - has statx                : no [2]
    - has statx syscall        : no [2]
    - cxx11_auto_declarations  : yes [2]
    - cxx11_constexpr          : yes [2]
    - cxx11_defaulted_functions : yes [2]
    - cxx11_final              : yes [2]
    - cxx11_hdr_mutex          : yes [2]
    - cxx11_hdr_tuple          : yes [2]
    - cxx11_lambdas            : yes [2]
    - cxx11_noexcept           : yes [2]
    - cxx11_nullptr            : yes [2]
    - cxx11_rvalue_references  : yes [2]
    - cxx11_template_aliases   : yes [2]
    - cxx11_thread_local       : no [2]
    - cxx11_variadic_templates : yes [2]
    - cxx11_auto_declarations  : yes [3]
    - cxx11_constexpr          : yes [3]
    - cxx11_defaulted_functions : yes [3]
    - cxx11_final              : yes [3]
    - cxx11_hdr_mutex          : yes [3]
    - cxx11_hdr_tuple          : yes [3]
    - cxx11_lambdas            : yes [3]
    - cxx11_noexcept           : yes [3]
    - cxx11_nullptr            : yes [3]
    - cxx11_rvalue_references  : yes [3]
    - cxx11_template_aliases   : yes [3]
    - cxx11_thread_local       : no [3]
    - cxx11_variadic_templates : yes [3]
    - has_icu builds           : no [2]
warning: Graph library does not contain MPI-based parallel components.
note: to enable them, add "using mpi ;" to your user-config.jam.
note: to suppress this message, pass "--without-graph_parallel" to bjam.
    - zlib                     : no [4]
    - bzip2                    : no [4]
    - lzma                     : no [4]
    - zstd                     : no [4]
    - has_lzma_cputhreads builds : no [2]
    - cxx11_decltype           : yes [2]
    - cxx11_basic_alignas      : yes [2]
    - iconv (libc)             : no [2]
    - iconv (separate)         : no [2]
    - icu                      : no [2]
    - icu (lib64)              : no [2]
    - native atomic int32 supported : yes [2]
    - has message compiler     : no [2]
    - native syslog supported  : no [2]
    - pthread supports robust mutexes : no [2]
    - compiler supports SSSE3  : yes [2]
    - compiler supports AVX2   : yes [2]
    - gcc visibility           : yes [2]
    - sfinae_expr              : yes [2]
    - cxx11_unified_initialization_syntax : yes [2]
    - cxx11_hdr_initializer_list : yes [2]
    - cxx11_hdr_chrono         : yes [2]
    - cxx11_numeric_limits     : yes [2]
    - cxx11_hdr_array          : yes [2]
    - cxx11_hdr_atomic         : yes [2]
    - cxx11_hdr_type_traits    : yes [2]
    - cxx11_allocator          : yes [2]
    - cxx11_explicit_conversion_operators : yes [2]
    - gcc visibility           : yes [5]
    - cxx11_noexcept           : yes [5]
    - cxx11_rvalue_references  : yes [5]
    - sfinae_expr              : yes [5]
    - cxx11_auto_declarations  : yes [5]
    - cxx11_lambdas            : yes [5]
    - cxx11_unified_initialization_syntax : yes [5]
    - cxx11_hdr_tuple          : yes [5]
    - cxx11_hdr_initializer_list : yes [5]
    - cxx11_hdr_chrono         : yes [5]
    - cxx11_thread_local       : no [5]
    - cxx11_constexpr          : yes [5]
    - cxx11_nullptr            : yes [5]
    - cxx11_numeric_limits     : yes [5]
    - cxx11_decltype           : yes [5]
    - cxx11_hdr_array          : yes [5]
    - cxx11_hdr_atomic         : yes [5]
    - cxx11_hdr_type_traits    : yes [5]
    - cxx11_allocator          : yes [5]
    - cxx11_explicit_conversion_operators : yes [5]
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
    - cxx11_static_assert      : yes [2]
    - std::fstream is moveable and swappable : yes [2]
    - Has Large File Support   : no [2]
    - libbacktrace builds      : no [2]
    - libbacktrace builds      : no [3]
    - addr2line builds         : no [2]
    - addr2line builds         : no [3]
    - WinDbg builds            : no [2]
    - WinDbg builds            : no [3]
    - WinDbgCached builds      : no [2]
    - WinDbgCached builds      : no [3]
    - BOOST_COMP_GNUC >= 4.3.0 : yes [2]
    - compiler supports SSE2   : yes [6]
    - compiler supports SSE4.1 : yes [6]
    - has synchronization.lib  : yes [6]
    - C++11 mutex              : yes [6]
    - has BCrypt API           : yes [6]
    - has stat::st_mtim        : no [6]
    - has stat::st_mtimensec   : no [6]
    - has stat::st_mtimespec   : no [6]
    - has stat::st_birthtim    : no [6]
    - has stat::st_birthtimensec : no [6]
    - has stat::st_birthtimespec : no [6]
    - has statx                : no [6]
    - has statx syscall        : no [6]
    - cxx11_auto_declarations  : yes [6]
    - cxx11_constexpr          : yes [6]
    - cxx11_defaulted_functions : yes [6]
    - cxx11_final              : yes [6]
    - cxx11_hdr_mutex          : yes [6]
    - cxx11_hdr_tuple          : yes [6]
    - cxx11_lambdas            : yes [6]
    - cxx11_noexcept           : yes [6]
    - cxx11_nullptr            : yes [6]
    - cxx11_rvalue_references  : yes [6]
    - cxx11_template_aliases   : yes [6]
    - cxx11_thread_local       : yes [6]
    - cxx11_variadic_templates : yes [6]
    - cxx11_auto_declarations  : yes [7]
    - cxx11_constexpr          : yes [7]
    - cxx11_defaulted_functions : yes [7]
    - cxx11_final              : yes [7]
    - cxx11_hdr_mutex          : yes [7]
    - cxx11_hdr_tuple          : yes [7]
    - cxx11_lambdas            : yes [7]
    - cxx11_noexcept           : yes [7]
    - cxx11_nullptr            : yes [7]
    - cxx11_rvalue_references  : yes [7]
    - cxx11_template_aliases   : yes [7]
    - cxx11_thread_local       : yes [7]
    - cxx11_variadic_templates : yes [7]
    - has_icu builds           : no [6]
    - zlib                     : no  (cached) [4]
    - bzip2                    : no  (cached) [4]
    - lzma                     : no  (cached) [4]
    - zstd                     : no  (cached) [4]
    - has_lzma_cputhreads builds : no [6]
    - cxx11_decltype           : yes [6]
    - cxx11_basic_alignas      : yes [6]
    - iconv (libc)             : no [6]
    - iconv (separate)         : yes [6]
    - icu                      : no [6]
    - icu (lib64)              : no [6]
    - native atomic int32 supported : yes [6]
    - has message compiler     : no [6]
    - native syslog supported  : no [6]
    - pthread supports robust mutexes : no [6]
    - compiler supports SSSE3  : yes [6]
    - compiler supports AVX2   : yes [6]
    - gcc visibility           : yes [6]
    - sfinae_expr              : yes [6]
    - cxx11_unified_initialization_syntax : yes [6]
    - cxx11_hdr_initializer_list : yes [6]
    - cxx11_hdr_chrono         : yes [6]
    - cxx11_numeric_limits     : yes [6]
    - cxx11_hdr_array          : yes [6]
    - cxx11_hdr_atomic         : yes [6]
    - cxx11_hdr_type_traits    : yes [6]
    - cxx11_allocator          : yes [6]
    - cxx11_explicit_conversion_operators : yes [6]
    - gcc visibility           : yes [8]
    - cxx11_noexcept           : yes [8]
    - cxx11_rvalue_references  : yes [8]
    - sfinae_expr              : yes [8]
    - cxx11_auto_declarations  : yes [8]
    - cxx11_lambdas            : yes [8]
    - cxx11_unified_initialization_syntax : yes [8]
    - cxx11_hdr_tuple          : yes [8]
    - cxx11_hdr_initializer_list : yes [8]
    - cxx11_hdr_chrono         : yes [8]
    - cxx11_thread_local       : yes [8]
    - cxx11_constexpr          : yes [8]
    - cxx11_nullptr            : yes [8]
    - cxx11_numeric_limits     : yes [8]
    - cxx11_decltype           : yes [8]
    - cxx11_hdr_array          : yes [8]
    - cxx11_hdr_atomic         : yes [8]
    - cxx11_hdr_type_traits    : yes [8]
    - cxx11_allocator          : yes [8]
    - cxx11_explicit_conversion_operators : yes [8]
    - long double support      : yes [8]
    - cxx11_static_assert      : yes [6]
    - std::fstream is moveable and swappable : yes [6]
    - Has Large File Support   : yes [6]
    - libbacktrace builds      : no [6]
    - libbacktrace builds      : no [7]
    - addr2line builds         : no [6]
    - addr2line builds         : no [7]
    - WinDbg builds            : yes [6]
    - WinDbg builds            : yes [7]
    - WinDbgCached builds      : yes [6]
    - BOOST_COMP_GNUC >= 4.3.0 : yes [6]

[1] gcc-10.2.0
[2] gcc-10.2.0/release/address-model-32/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[3] gcc-10.2.0/release/address-model-32/build-no/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[4] link-static
[5] gcc-10.2.0/release/address-model-32/build-no/link-static/pch-off/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[6] gcc-10.2.0/release/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[7] gcc-10.2.0/release/build-no/link-static/python-3.8/threadapi-win32/threading-multi/visibility-hidden
[8] gcc-10.2.0/release/link-static/pch-off/python-3.8/threadapi-win32/threading-multi/visibility-hidden

Component configuration:

    - atomic                   : building
    - chrono                   : building
    - container                : building
    - context                  : building
    - contract                 : building
    - coroutine                : building
    - date_time                : building
    - exception                : building
    - fiber                    : building
    - filesystem               : building
    - graph                    : building
    - graph_parallel           : building
    - headers                  : building
    - iostreams                : building
    - json                     : building
    - locale                   : building
    - log                      : building
    - math                     : building
    - mpi                      : building
    - nowide                   : building
    - program_options          : building
    - python                   : building
    - random                   : building
    - regex                    : building
    - serialization            : building
    - stacktrace               : building
    - system                   : building
    - test                     : building
    - thread                   : building
    - timer                    : building
    - type_erasure             : building
    - wave                     : building

...found 34574 targets...
...updating 17670 targets...
...patience...
[... shortened - the whole Log is too big ...]

If you want the whole Log I can upload it. Just let me know. It is 8.1 Megabyte of Text.

Expected behavior summary

The Header File 'pyconfig.h' should get detected correctly and the Boost Libraries should get compiled without further Issues.

Disclaimer: This is my first Day with Boost at all. I just need to build some Libs for some GTKMM4 Dependencies. And as I need to compile from Source anyway, to get Build Variants not provided by MSYS2, here I am reporting an 9 year old Issue.

adeadman commented 2 years ago

This might be related to boostorg/build#481 where the Python installation is built with PyMalloc, so the header path has an m suffix (e.g. path/to/Python/3.8.x/include/python3.8m)

MeerMusik commented 2 years ago

Wow. I totally forgot about that Bug Report. Thanks for the Hint adeadman!

A general FYI for People interested: Since I have reported this Bug, I am building Boost on Linux and Windows always with: --without-python as a work-around.

This does not change the Detection of Python as it is hard-coded in the configuration Files but compilation works fine.

I am not using the Boost Libs currently, so I am not aware of the Cons of not building with / against Python support.

Have a nice Day.

StaySafeStayHealthyEveryone

rlovell3 commented 1 year ago

I needed to build the Boost.Python libraries back in 2012 or so, and it was an absolute nightmare back then. Now, here in 2023, I'm having to do it again, and having the same b2 and jam headaches. Even ChatGPT can't get it to compile.... All we really need is better documentation. For example, I'm running into the age-old problem of the compiler complaining about not finding Python.h. Capital P in Python.h, at least on this Linux machine. I'm an extensive user of conda environments, and I'm trying to bootstrap and b2 off of one of my environments. I've burned through 3 full days now trying to get the config right, and there seems to be no hope. Of all the code I have written over the decades of writing code, nothing has ever been as completely frustrating as trying to build Boost.Python with bootstrap and b2. I'm hoping that by venting here, I go back to my build project and it magically compiles so I can get on with what I need to be doing.

rlovell3 commented 1 year ago

For what it is worth, as is often the case, after ranting publicly, I quickly came up with a solution that had been confounding me for days. My notes, should anyone care are here: github.com/rlovell3/boost_python_compile_notes. Those notes are for building on Ubuntu and associated flavors, using a highly customized conda environment running Python 3.11. Thank you to all the Boost contributors. Boost is truly an amazing work. b2 is still a pain though, but I get it.