search

oracle / graalpython

GraalPy – A high-performance embeddable Python 3 runtime for Java
https://www.graalvm.org/python/
Other
1.24k stars 108 forks source link

pip install cmake fails #262

Closed d-leroy closed 1 year ago

d-leroy commented 2 years ago

As the title says, pip install cmake in an activated graalpython venv fails with the following output:

Collecting cmake
  WARNING: Cache entry deserialization failed, entry ignored
  Downloading cmake-3.22.4.tar.gz (31 kB)
Looking for Graal Python patches for cmake
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /home/leroyd/venv-graalvm/bin/graalpython /home/leroyd/venv-graalvm/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpy3ss3k36
       cwd: /tmp/pip-install-59vrd70q/cmake
  Complete output (97 lines):
  Traceback (most recent call last):
    "<internal>", line 1, in <module>
    File "_in_process.py", line 280, in <module '_in_process.py'>
    File "_in_process.py", line 263, in main
    File "_in_process.py", line 114, in get_requires_for_build_wheel
    File "build_meta.py", line 177, in get_requires_for_build_wheel
    File "build_meta.py", line 159, in _get_build_requires
    File "build_meta.py", line 174, in run_setup
    File "setup.py", line 8, in <module 'setup.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "__init__.py", line 9, in <module '__init__.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "setuptools_wrap.py", line 36, in <module 'setuptools_wrap.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "requirements.py", line 10, in <module 'requirements.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "__init__.py", line 144, in <module '__init__.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "helpers.py", line 686, in <module 'helpers.py'>
    File "core.py", line 2717, in __init__
  list does not support iterable object ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_:
  Traceback (most recent call last):
    "<internal>", line 1, in <module>
    File "_in_process.py", line 280, in <module '_in_process.py'>
    File "_in_process.py", line 263, in main
    File "_in_process.py", line 114, in get_requires_for_build_wheel
    File "build_meta.py", line 177, in get_requires_for_build_wheel
    File "build_meta.py", line 159, in _get_build_requires
    File "build_meta.py", line 174, in run_setup
    File "setup.py", line 8, in <module 'setup.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "__init__.py", line 9, in <module '__init__.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "setuptools_wrap.py", line 36, in <module 'setuptools_wrap.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "requirements.py", line 10, in <module 'requirements.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "__init__.py", line 144, in <module '__init__.py'>
    "/importlib/_bootstrap.py", line 991, in _find_and_load
    "/importlib/_bootstrap.py", line 975, in _find_and_load_unlocked
    "/importlib/_bootstrap.py", line 671, in _load_unlocked
    "/importlib/_bootstrap_external.py", line 788, in exec_module
    "/importlib/_bootstrap.py", line 219, in _call_with_frames_removed
    File "helpers.py", line 686, in <module 'helpers.py'>
    File "core.py", line 2717, in __init__
  list does not support iterable object ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_:
  Traceback (most recent call last):
    File "_in_process.py", line 280, in <module '_in_process.py'>
    File "_in_process.py", line 263, in main
    File "_in_process.py", line 114, in get_requires_for_build_wheel
    File "build_meta.py", line 177, in get_requires_for_build_wheel
    File "build_meta.py", line 159, in _get_build_requires
    File "build_meta.py", line 174, in run_setup
    File "setup.py", line 8, in <module 'setup.py'>
    File "__init__.py", line 9, in <module '__init__.py'>
    File "setuptools_wrap.py", line 36, in <module 'setuptools_wrap.py'>
    File "requirements.py", line 10, in <module 'requirements.py'>
    File "__init__.py", line 144, in <module '__init__.py'>
    File "helpers.py", line 686, in <module 'helpers.py'>
    File "core.py", line 2717, in __init__
  com.oracle.truffle.api.CompilerDirectives$ShouldNotReachHere: list does not support iterable object ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_:
d-leroy commented 2 years ago

Hi, I tried it again on the new GraalVM community version (22.1.0), and got a different error message:

Building wheels for collected packages: cmake
  Created temporary directory: /tmp/pip-wheel-qbudyufc
  Destination directory: /tmp/pip-wheel-qbudyufc
  Running command /home/leroyd/venv-graalvm/bin/graalpython /home/leroyd/venv-graalvm/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp9i_r0ghh

  --------------------------------------------------------------------------------
  -- Trying "Ninja" generator
  --------------------------------
  ---------------------------
  ----------------------
  -----------------
  ------------
  -------
  --
  Not searching for unused variables given on the command line.
  -- The C compiler identification is Clang 12.0.1
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /home/leroyd/venv-graalvm/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- The CXX compiler identification is Clang 12.0.1
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /home/leroyd/venv-graalvm/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /tmp/pip-install-wsd4r886/cmake/_cmake_test_compile/build
  --
  -------
  ------------
  -----------------
  ----------------------
  ---------------------------
  --------------------------------
  -- Trying "Ninja" generator - success
  --------------------------------------------------------------------------------

  Traceback (most recent call last):
    File "/home/leroyd/venv-graalvm/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module '_in_process.py'>
      main()
    File "/home/leroyd/venv-graalvm/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/leroyd/venv-graalvm/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-4gkcdsou/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 244, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/tmp/pip-build-env-4gkcdsou/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 229, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-4gkcdsou/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "/tmp/pip-install-wsd4r886/cmake/setup.py", line 32, in <module 'setup.py'>
      setup(
    File "/tmp/pip-build-env-4gkcdsou/overlay/lib/python3.8/site-packages/skbuild/setuptools_wrap.py", line 637, in setup
      env = cmkr.configure(
    File "/tmp/pip-build-env-4gkcdsou/overlay/lib/python3.8/site-packages/skbuild/cmaker.py", line 273, in configure
      ("-DPYTHON_INCLUDE_DIR:PATH=" + python_include_dir),
  TypeError: can only concatenate str (not "NoneType") to str
  Building wheel for cmake (PEP 517) ... error
  ERROR: Failed building wheel for cmake

After looking at cmaker.py, it seems skbuild is looking for the 'INCLUDEPY' variable in sysconfig but the only variable defined on my installation pointing to the include directory of Python is 'CONFINCLUDEPY'.

Is there maybe a workaround to force a value in sysconfig for 'INCLUDEPY'?

msimacek commented 2 years ago

Hi, I already have a PR that adds the missing sysconfig variables, but unfortunately it's not enough to make the cmake build succeed. It now fails because some C++ binary links against our C++ standard library, but cannot find it at runtime.

msimacek commented 1 year ago

I couldn't get cmake built with our toolchain, so I added a patch that makes it download prebuilt cmake. Now it should install fine.