[llvm] build failure with feature [enable-mlir-python-bindings]

ankurvdev commented 9 months ago

Operating system

fedora-39 (linux)

Compiler

gcc-13

Steps to reproduce the behavior

`./vcpkg install 'llvm[enable-mlir-python-bindings]'`

Failure logs

issue_body.md -- Looking for futimens - found -- Looking for futimes -- Looking for futimes - found -- Looking for sigaltstack -- Looking for sigaltstack - found -- Looking for mallctl -- Looking for mallctl - not found -- Looking for mallinfo -- Looking for mallinfo - found -- Looking for mallinfo2 -- Looking for mallinfo2 - found -- Looking for malloc_zone_statistics -- Looking for malloc_zone_statistics - not found -- Looking for getrlimit -- Looking for getrlimit - found -- Looking for posix_spawn -- Looking for posix_spawn - found -- Looking for pread -- Looking for pread - found -- Looking for sbrk -- Looking for sbrk - found -- Looking for strerror_r -- Looking for strerror_r - found -- Looking for strerror_s -- Looking for strerror_s - not found -- Looking for setenv -- Looking for setenv - found -- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC -- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC - Failed -- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC -- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC - Success -- Looking for GLIBC -- Looking for GLIBC - found -- Looking for pthread_getname_np -- Looking for pthread_getname_np - found -- Looking for pthread_setname_np -- Looking for pthread_setname_np - found -- Looking for dlopen -- Looking for dlopen - found -- Looking for dladdr -- Looking for dladdr - found -- Looking for proc_pid_rusage -- Looking for proc_pid_rusage - not found -- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB -- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB - Success -- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB -- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB - Success -- Performing Test LLVM_HAS_ATOMICS -- Performing Test LLVM_HAS_ATOMICS - Success -- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG -- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG - Success -- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG -- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG - Failed -- Performing Test HAS_MAYBE_UNINITIALIZED -- Performing Test HAS_MAYBE_UNINITIALIZED - Success -- Native target architecture is X86 -- Threads enabled. -- Doxygen disabled. -- Ninja version: 1.11.1 -- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) -- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH) -- OCaml bindings disabled. -- Could NOT find Python module pygments -- Could NOT find Python module pygments.lexers.c_cpp -- Found Python module yaml -- LLVM host triple: x86_64-unknown-linux-gnu -- LLVM default target triple: x86_64-unknown-linux-gnu -- Performing Test C_SUPPORTS_FPIC -- Performing Test C_SUPPORTS_FPIC - Success -- Performing Test CXX_SUPPORTS_FPIC -- Performing Test CXX_SUPPORTS_FPIC - Success -- Building with -fPIC -- Performing Test C_SUPPORTS_FNO_SEMANTIC_INTERPOSITION -- Performing Test C_SUPPORTS_FNO_SEMANTIC_INTERPOSITION - Success -- Performing Test CXX_SUPPORTS_FNO_SEMANTIC_INTERPOSITION -- Performing Test CXX_SUPPORTS_FNO_SEMANTIC_INTERPOSITION - Success -- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG -- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG - Success -- Performing Test C_SUPPORTS_WERROR_DATE_TIME -- Performing Test C_SUPPORTS_WERROR_DATE_TIME - Success -- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME -- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME - Success -- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW -- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW - Failed -- Performing Test C_SUPPORTS_CMAKE_CXX_FLAGS -- Performing Test C_SUPPORTS_CMAKE_CXX_FLAGS - Success -- Performing Test CXX_SUPPORTS_CMAKE_CXX_FLAGS -- Performing Test CXX_SUPPORTS_CMAKE_CXX_FLAGS - Success -- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG -- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG - Success -- Performing Test C_SUPPORTS_CXX98_COMPAT_EXTRA_SEMI_FLAG -- Performing Test C_SUPPORTS_CXX98_COMPAT_EXTRA_SEMI_FLAG - Failed -- Performing Test CXX_SUPPORTS_CXX98_COMPAT_EXTRA_SEMI_FLAG -- Performing Test CXX_SUPPORTS_CXX98_COMPAT_EXTRA_SEMI_FLAG - Failed -- Performing Test C_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG -- Performing Test C_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG - Success -- Performing Test CXX_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG -- Performing Test CXX_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG - Success -- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG -- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed -- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG -- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed -- Performing Test CXX_SUPPORTS_CLASS_MEMACCESS_FLAG -- Performing Test CXX_SUPPORTS_CLASS_MEMACCESS_FLAG - Success -- Performing Test CXX_SUPPORTS_REDUNDANT_MOVE_FLAG -- Performing Test CXX_SUPPORTS_REDUNDANT_MOVE_FLAG - Success -- Performing Test CXX_SUPPORTS_PESSIMIZING_MOVE_FLAG -- Performing Test CXX_SUPPORTS_PESSIMIZING_MOVE_FLAG - Success -- Performing Test CXX_SUPPORTS_NOEXCEPT_TYPE_FLAG -- Performing Test CXX_SUPPORTS_NOEXCEPT_TYPE_FLAG - Success -- Performing Test CXX_WONT_WARN_ON_FINAL_NONVIRTUALDTOR -- Performing Test CXX_WONT_WARN_ON_FINAL_NONVIRTUALDTOR - Failed -- Performing Test CXX_SUPPORTS_SUGGEST_OVERRIDE_FLAG -- Performing Test CXX_SUPPORTS_SUGGEST_OVERRIDE_FLAG - Success -- Performing Test CXX_WSUGGEST_OVERRIDE_ALLOWS_ONLY_FINAL -- Performing Test CXX_WSUGGEST_OVERRIDE_ALLOWS_ONLY_FINAL - Success -- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP -- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP - Failed -- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG -- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG - Failed -- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG -- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG - Failed -- Performing Test CXX_SUPPORTS_MISLEADING_INDENTATION_FLAG -- Performing Test CXX_SUPPORTS_MISLEADING_INDENTATION_FLAG - Success -- Performing Test C_SUPPORTS_CTAD_MAYBE_UNSPPORTED_FLAG -- Performing Test C_SUPPORTS_CTAD_MAYBE_UNSPPORTED_FLAG - Failed -- Performing Test CXX_SUPPORTS_CTAD_MAYBE_UNSPPORTED_FLAG -- Performing Test CXX_SUPPORTS_CTAD_MAYBE_UNSPPORTED_FLAG - Success -- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS -- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS - Failed -- Performing Test C_SUPPORTS_FNO_FUNCTION_SECTIONS -- Performing Test C_SUPPORTS_FNO_FUNCTION_SECTIONS - Success -- Performing Test C_SUPPORTS_FFUNCTION_SECTIONS -- Performing Test C_SUPPORTS_FFUNCTION_SECTIONS - Success -- Performing Test CXX_SUPPORTS_FFUNCTION_SECTIONS -- Performing Test CXX_SUPPORTS_FFUNCTION_SECTIONS - Success -- Performing Test C_SUPPORTS_FDATA_SECTIONS -- Performing Test C_SUPPORTS_FDATA_SECTIONS - Success -- Performing Test CXX_SUPPORTS_FDATA_SECTIONS -- Performing Test CXX_SUPPORTS_FDATA_SECTIONS - Success -- Looking for os_signpost_interval_begin -- Looking for os_signpost_interval_begin - not found -- Linker detection: GNU ld -- Performing Test HAS_WERROR_GLOBAL_CTORS -- Performing Test HAS_WERROR_GLOBAL_CTORS - Failed -- Looking for x86_64 -- Looking for x86_64 - found -- Found Git: /usr/bin/git (found version "2.43.0") -- Targeting X86 CMake Deprecation Warning at /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/cmake/Modules/CMakePolicy.cmake:6 (cmake_policy): The OLD behavior for policy CMP0114 will be removed from a future version of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all policies are deprecated and that a policy should be set to OLD only under specific short-term circumstances. Projects should be ported to the NEW behavior and not rely on setting a policy to OLD. Call Stack (most recent call first): /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/clang/CMakeLists.txt:6 (include)

-- Looking for sys/resource.h -- Looking for sys/resource.h - found -- Clang version: 17.0.2 -- Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG -- Performing Test CXX_SUPPORTS_NO_NESTED_ANON_TYPES_FLAG - Failed -- Looking for include file sys/inotify.h -- Looking for include file sys/inotify.h - found CMake Deprecation Warning at /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/cmake/Modules/CMakePolicy.cmake:6 (cmake_policy): The OLD behavior for policy CMP0114 will be removed from a future version of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all policies are deprecated and that a policy should be set to OLD only under specific short-term circumstances. Projects should be ported to the NEW behavior and not rely on setting a policy to OLD. Call Stack (most recent call first): /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/lld/CMakeLists.txt:6 (include)

-- LLD version: 17.0.2 CMake Deprecation Warning at /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/cmake/Modules/CMakePolicy.cmake:6 (cmake_policy): The OLD behavior for policy CMP0114 will be removed from a future version of CMake.

The cmake-policies(7) manual explains that the OLD behaviors of all policies are deprecated and that a policy should be set to OLD only under specific short-term circumstances. Projects should be ported to the NEW behavior and not rely on setting a policy to OLD. Call Stack (most recent call first): /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/mlir/CMakeLists.txt:7 (include)

-- Performing Test C_SUPPORTS_WERROR_IMPLICIT_FUNCTION_DECLARATION -- Performing Test C_SUPPORTS_WERROR_IMPLICIT_FUNCTION_DECLARATION - Success -- Performing Test C_SUPPORTS_WERROR_MISMATCHED_TAGS -- Performing Test C_SUPPORTS_WERROR_MISMATCHED_TAGS - Failed -- Could NOT find Python3 (missing: Development Development.Module Development.Embed) (found suitable version "3.12.0", minimum required is "3.0") CMake Error at /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:230 (message): Could NOT find Python3 (missing: Development.Module) (found suitable version "3.12.0", minimum required is "3.0") Call Stack (most recent call first): /usr/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE) /usr/share/cmake/Modules/FindPython/Support.cmake:3824 (find_package_handle_standard_args) /usr/share/cmake/Modules/FindPython3.cmake:545 (include) /home/ankurv/vcpkg.ort/installed/x64-linux/share/python3/vcpkg-cmake-wrapper.cmake:67 (_find_package) /home/ankurv/vcpkg.ort/scripts/buildsystems/vcpkg.cmake:813 (include) /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/mlir/cmake/modules/MLIRDetectPythonEnv.cmake:21 (find_package) /home/ankurv/vcpkg.ort/buildtrees/llvm/src/org-17.0.2-1f7ed5ae6e.clean/mlir/CMakeLists.txt:156 (mlir_configure_python_dev_packages)

-- Configuring incomplete, errors occurred! ninja: build stopped: subcommand failed.



### Additional context

The issue seems to be due to mismatch between vcpkg's python version (3.11) and the the installed python version (3.12)

For some reason the installed interpreter is used but the Development.Module libraries available are only 3.11

I do have python3-devel rpm installed on my environment so either
1. vcpkg should choose the interpreter and libraries from vcpkg/installed
or 
2. vcpkg should choose the interpreter and libraries from the os environment.

The mix and match of interpreter and vcpkg-libraries seems  causing this error to happen

ankurvdev commented 9 months ago

The issue seems to be due to mismatch between vcpkg's python version (3.11) and the the installed python version (3.12)

For some reason the installed interpreter is used but the Development.Module libraries available are only 3.11

I do have python3-devel rpm installed on my environment so either

vcpkg should choose the interpreter and libraries from vcpkg/installed or
vcpkg should choose the interpreter and libraries from the os environment.

The mix and match of interpreter and vcpkg-libraries seems causing this error to happen

ankurvdev commented 9 months ago

a temporary workaround is to remove "python3" from vcpkg.json listed as "enable-mlir-python-bindings" feature dependency and remove python3 from vcpkg installed list so it can use os libraries

JonLiu1993 commented 9 months ago

Confirmed! This issue could be reproduced with vcpkg 2023-11-16-4c1df40a3c5c5e18de299a99e9accb03c2a82e1e.

ankurvdev commented 4 months ago

any update ?

microsoft / vcpkg