search

mesonbuild / meson-python

Meson PEP 517 Python build backend
https://mesonbuild.com/meson-python/
MIT License
118 stars 59 forks source link

building scipy in VS code Docker Dev ubuntu container: No such file or directory: '/tmp/pip-build-env-na93e837/overlay/bin/pybind11-config' #645

Closed lobpcg closed 5 days ago

lobpcg commented 5 days ago

See https://github.com/scipy/scipy/issues/21040

Describe your issue.

In the latest VS Code in Win 11, I created a container with scipy in it, and use the (scipy-dev) conda environment. Building scipy with meson gives the error FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-build-env-na93e837/overlay/bin/pybind11-config' but there is no even such directory:

cd /tmp/pip-build-env-na93e837/
bash: cd: /tmp/pip-build-env-na93e837/: No such file or directory

while 

(scipy-dev) codespace ➜ /workspaces/scipy (main) $ pip install "pybind11[global]"
Requirement already satisfied: pybind11[global] in /home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages (2.12.0)
Requirement already satisfied: pybind11-global==2.12.0 in /home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages (from pybind11[global]) (2.12.0)
(scipy-dev) codespace ➜ /workspaces/scipy (main) $ python -m pip install --no-build-isolation 
--config-settings=editable-verbose=true --editable .
Obtaining file:///workspaces/scipy
  Checking if build backend supports build_editable ... done
  Preparing editable metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Preparing editable metadata (pyproject.toml) did not run successfully.
  │ exit code: 2
  ╰─> [192 lines of output]
      + meson setup --reconfigure /workspaces/scipy /workspaces/scipy/build/cp312 -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/workspaces/scipy/build/cp312/meson-python-native-file.ini
      Traceback (most recent call last):
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/mesonmain.py", line 186, in run
          return options.run_func(options)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/msetup.py", line 363, in run
          app.generate()
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/msetup.py", line 187, in generate
          return self._generate(env, capture, vslite_ctx)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/msetup.py", line 226, in _generate
          intr.run()
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 3011, in run
          super().run()
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 162, in run
          self.evaluate_codeblock(self.ast, start=1)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 188, in evaluate_codeblock
          raise e
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 180, in evaluate_codeblock
          self.evaluate_statement(cur)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 194, in evaluate_statement
          return self.function_call(cur)
                 ^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 520, in function_call
          res = func(node, func_args, kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 250, in wrapper
          return f(*nargs, **wrapped_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 569, in wrapper
          return f(*wrapped_args, **wrapped_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreter/interpreter.py", line 2429, in func_subdir
          self.evaluate_codeblock(codeblock)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 188, in evaluate_codeblock
          raise e
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 180, in evaluate_codeblock
          self.evaluate_statement(cur)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 233, in evaluate_statement
          self.evaluate_foreach(cur)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 471, in evaluate_foreach
          self.evaluate_codeblock(node.block)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 188, in evaluate_codeblock
          raise e
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 180, in evaluate_codeblock
          self.evaluate_statement(cur)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 211, in evaluate_statement
          return self.evaluate_if(cur)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 301, in evaluate_if
          self.evaluate_codeblock(i.block)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 188, in evaluate_codeblock
          raise e
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 180, in evaluate_codeblock
          self.evaluate_statement(cur)
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
          return self.method_call(cur)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 536, in method_call
          (h_args, h_kwargs) = self.reduce_arguments(node.args)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 592, in reduce_arguments
          reduced_pos = [self.evaluate_statement(arg) for arg in args.arguments]
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
          return self.method_call(cur)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/interpreterbase.py", line 549, in method_call
          res = obj.method_call(method_name, args, kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/baseobjects.py", line 81, in method_call
          return method(args, kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 663, in wrapped
          return f(*wrapped_args, **wrapped_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 250, in wrapper
          return f(*nargs, **wrapped_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreterbase/decorators.py", line 569, in wrapper
          return f(*wrapped_args, **wrapped_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/interpreter/interpreterobjects.py", line 553, in variable_method
          return self.held_object.get_variable(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/dependencies/configtool.py", line 156, in get_variable
          p, out, _ = Popen_safe(self.config + self.get_variable_args(configtool))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/site-packages/mesonbuild/utils/universal.py", line 1521, in Popen_safe
          p = subprocess.Popen(args, universal_newlines=True, encoding=encoding, close_fds=False,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/subprocess.py", line 1026, in __init__
          self._execute_child(args, executable, preexec_fn, close_fds,
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/subprocess.py", line 1840, in _execute_child
          self._posix_spawn(args, executable, env, restore_signals,
        File "/home/codespace/micromamba/envs/scipy-dev/lib/python3.12/subprocess.py", line 1784, in _posix_spawn
          self.pid = os.posix_spawn(executable, args, env, **kwargs)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-build-env-na93e837/overlay/bin/pybind11-config'
      The Meson build system
      Version: 1.4.1
      Source dir: /workspaces/scipy
      Build dir: /workspaces/scipy/build/cp312
      Build type: native build
      Project name: scipy
      Project version: 1.15.0.dev0
      C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0")
      C linker for the host machine: cc ld.bfd 2.34
      C++ compiler for the host machine: c++ (gcc 9.4.0 "c++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0")
      C++ linker for the host machine: c++ ld.bfd 2.34
      Cython compiler for the host machine: cython (cython 3.0.10)
      Host machine cpu family: x86_64
      Host machine cpu: x86_64
      Program python found: YES (/home/codespace/micromamba/envs/scipy-dev/bin/python)
      Found pkg-config: YES (/home/codespace/micromamba/envs/scipy-dev/bin/pkg-config) 0.29.2
      Run-time dependency python found: YES 3.12
      Program cython found: YES (/home/codespace/micromamba/envs/scipy-dev/bin/cython)
      Compiler for C supports arguments -Wno-unused-but-set-variable: YES (cached)
      Compiler for C supports arguments -Wno-unused-function: YES (cached)
      Compiler for C supports arguments -Wno-conversion: YES (cached)
      Compiler for C supports arguments -Wno-misleading-indentation: YES (cached)
      Library m found: YES
      Fortran compiler for the host machine: gfortran (gcc 9.4.0 "GNU Fortran (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0")
      Fortran linker for the host machine: gfortran ld.bfd 2.34
      Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
      Checking if "-Wl,--version-script" : links: YES (cached)
      Program tools/generate_f2pymod.py found: YES (/home/codespace/micromamba/envs/scipy-dev/bin/python /workspaces/scipy/tools/generate_f2pymod.py)
      Program scipy/_build_utils/tempita.py found: YES (/home/codespace/micromamba/envs/scipy-dev/bin/python /workspaces/scipy/scipy/_build_utils/tempita.py)
      Program pythran found: YES 0.16.1 0.16.1 (/home/codespace/micromamba/envs/scipy-dev/bin/pythran)
      Found CMake: /usr/bin/cmake (3.16.3)
      DEPRECATION: CMake support for versions <3.17 is deprecated since Meson 0.62.0.
      |
      |   However, Meson was only able to find CMake 3.16.3.
      |
      |   Support for all CMake versions below 3.17.0 will be removed once
      |   newer CMake versions are more widely adopted. If you encounter
      |   any errors please try upgrading CMake to a newer version first.

      WARNING: CMake Toolchain: Failed to determine CMake compilers state
      Run-time dependency xsimd found: NO (tried pkgconfig and cmake)
      Dependency threads found: YES unknown (cached)
      Library npymath found: YES
      Dependency pybind11 found: YES 2.12.0 (cached)
      Program f2py found: YES (/home/codespace/.local/bin/f2py)
      Run-time dependency scipy-openblas found: NO (tried pkgconfig)
      Dependency openblas found: YES 0.3.8 (cached)
      Dependency openblas found: YES 0.3.8 (cached)
      Program ../tools/version_utils.py found: YES (/home/codespace/micromamba/envs/scipy-dev/bin/python /workspaces/scipy/scipy/../tools/version_utils.py)
      Compiler for C supports arguments -Wno-maybe-uninitialized: YES (cached)
      Compiler for C supports arguments -Wno-discarded-qualifiers: YES (cached)
      Compiler for C supports arguments -Wno-empty-body: YES (cached)
      Compiler for C supports arguments -Wno-implicit-function-declaration: YES (cached)
      Compiler for C supports arguments -Wno-parentheses: YES (cached)
      Compiler for C supports arguments -Wno-switch: YES (cached)
      Compiler for C supports arguments -Wno-unused-label: YES (cached)
      Compiler for C supports arguments -Wno-unused-result: YES (cached)
      Compiler for C supports arguments -Wno-unused-variable: YES (cached)
      Compiler for C++ supports arguments -Wno-bitwise-instead-of-logical: NO (cached)
      Compiler for C++ supports arguments -Wno-cpp: YES (cached)
      Compiler for C++ supports arguments -Wno-class-memaccess: YES (cached)
      Compiler for C++ supports arguments -Wno-deprecated-declarations: YES (cached)
      Compiler for C++ supports arguments -Wno-deprecated-builtins: NO (cached)
      Compiler for C++ supports arguments -Wno-format-truncation: YES (cached)
      Compiler for C++ supports arguments -Wno-non-virtual-dtor: YES (cached)
      Compiler for C++ supports arguments -Wno-sign-compare: YES (cached)
      Compiler for C++ supports arguments -Wno-switch: YES (cached)
      Compiler for C++ supports arguments -Wno-terminate: YES (cached)
      Compiler for C++ supports arguments -Wno-unused-but-set-variable: YES (cached)
      Compiler for C++ supports arguments -Wno-unused-function: YES (cached)
      Compiler for C++ supports arguments -Wno-unused-local-typedefs: YES (cached)
      Compiler for C++ supports arguments -Wno-unused-variable: YES (cached)
      Compiler for C++ supports arguments -Wno-int-in-bool-context: YES (cached)
      Compiler for Fortran supports arguments -Wno-argument-mismatch: YES (cached)
      Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
      Compiler for Fortran supports arguments -Wno-intrinsic-shadow: YES (cached)
      Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES (cached)
      Compiler for Fortran supports arguments -Wno-surprising: YES (cached)
      Compiler for Fortran supports arguments -Wno-uninitialized: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-label: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-variable: YES (cached)
      Compiler for Fortran supports arguments -Wno-tabs: YES (cached)
      Compiler for Fortran supports arguments -Wno-argument-mismatch: YES (cached)
      Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
      Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-label: YES (cached)
      Compiler for Fortran supports arguments -Wno-unused-variable: YES (cached)
      Compiler for Fortran supports arguments -Wno-tabs: YES (cached)
      Checking if "Check atomic builtins without -latomic" : links: YES (cached)

      ERROR: Unhandled python OSError. This is probably not a Meson bug, but an issue with your build environment.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
rgommers commented 5 days ago

This doesn't look familiar, I don't quite know what is going on there. This is unlikely to be a meson-python bug; more something between SciPy, Pybind11, and whatever is going on with how VS Code spins up a container.

To not treat the same bug report in two places, I'll close this issue and will reply on https://github.com/scipy/scipy/issues/21040.