search

mesonbuild / meson-python

Meson PEP 517 Python build backend
https://mesonbuild.com/meson-python/
MIT License
131 stars 70 forks source link

building sdist fails with CMake subproject #668

Open lucascolley opened 2 months ago

lucascolley commented 2 months ago

Traceback:

$ python -m build --no-isolation -x -Csetup-args="-Duse-pythran=false"
...
+ meson setup D:\a\scipy\scipy D:\a\scipy\scipy\.mesonpy-nmuol9tn -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Duse-pythran=false --native-file=D:\a\scipy\scipy\.mesonpy-nmuol9tn\meson-python-native-file.ini
...
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 373, in <module>
    main()
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 326, in build_sdist
    return backend.build_sdist(sdist_directory, config_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 1020, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 1061, in build_sdist
    with _project(config_settings) as project:
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 144, in __exit__
    next(self.gen)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 944, in _project
    with contextlib.ExitStack() as ctx:
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 601, in __exit__
    raise exc_details[1]
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 586, in __exit__
    if cb(*exc_details):
       ^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 943, in __exit__
    self.cleanup()
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 947, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  [Previous line repeated 1 more time]
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 636, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '.\\.mesonpy-nmuol9tn\\meson-private\\cmake_boost_math\\CMakeFiles\\CMakeScratch'

ERROR Backend subprocess exited when trying to invoke build_sdist

Full log: https://github.com/scipy/scipy/actions/runs/10884814141/job/30200964434?pr=21270

x-ref scipy/scipy#21270

lucascolley commented 2 months ago

After naive searches for shutil.rmtree here and in mesonbuild/meson, perhaps this is an issue for that repo instead.

lucascolley commented 2 months ago

@eli-schwartz do you have the permission to transfer this issue to the other repo? (if that looks like the right idea here?)

dnicolodi commented 2 months ago

The stack trace points to the code in meson-python that cleans up the temporary build directory, thus this does not look to be a meson issue. However, I do not have any idea of what is keeping files in the build directory busy, except something related to meson. Windows is so weird in this regard that I don't even know where to start looking and I don't have easy access to a system where I can test this. Someone would need to debug this.

EDIT: The interesting part of the stack trace has been cut off from the report, but it is available in the CI log. Here is the whole thing:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 373, in <module>
    main()
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 357, in main
    json_out["return_val"] = hook(**hook_input["kwargs"])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 326, in build_sdist
    return backend.build_sdist(sdist_directory, config_settings)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 1020, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 1061, in build_sdist
    with _project(config_settings) as project:
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 144, in __exit__
    next(self.gen)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\site-packages\mesonpy\__init__.py", line 944, in _project
    with contextlib.ExitStack() as ctx:
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 601, in __exit__
    raise exc_details[1]
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\contextlib.py", line 586, in __exit__
    if cb(*exc_details):
       ^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 943, in __exit__
    self.cleanup()
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 947, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  [Previous line repeated 1 more time]
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\shutil.py", line 636, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '.\\.mesonpy-nmuol9tn\\meson-private\\cmake_boost_math\\CMakeFiles\\CMakeScratch'
lucascolley commented 2 months ago

oops, thanks for catching the full traceback! I've edited the top post

lucascolley commented 2 months ago

I do not have any idea of what is keeping files in the build directory busy, except something related to meson.

If I understand you correctly, it looks like a process related to building the CMake subproject introduced in that PR:

.\\.mesonpy-nmuol9tn\\meson-private\\cmake_boost_math\\CMakeFiles\\CMakeScratch
dnicolodi commented 2 months ago

Not building but just configuring. Sub-projects are not build, only configured, to create an sdist. I saw the path in the stack trace. But I have no idea what may keep that file busy. The only process executed by meson-python is meson. And meson is a single process Python application, thus parts of it are for sure not lingering around. For configuring a CMake subproject, meson executes cmake, but I'm pretty sure it waits for the child process to exit before continuing. Thus it is either a process started by CMake that stays around, or it is some weird Windows thing where something interposes between the process opening the file and the filesystem.

eli-schwartz commented 2 months ago

Well, cmake's normal means of operation when configuring a cmake build, is for the cmake equivalent of meson's cc.links() to produce a tiny ninja/make project in ${CMAKE_BINARY_DIR}/CMakeFiles/CMakeScratch/<random_name> and then try building that and checking if the ninja/make project succeeds.

This may be a recursion issue -- but I have no idea what or how cmake waits on its own subprocesses. Perhaps a sub-cmake is still lingering?

CMakeScratch is a definite red flag though because a directory filename as specific as that is absolutely 100% a cmake internal and meson doesn't touch it either. And cmake claims it cleans that up automatically as part of its own configure stage before exiting.

lucascolley commented 2 months ago

@mwoehlke-kitware I noticed that you touched the following lines. Perhaps you could shed some light here? https://github.com/Kitware/CMake/blob/master/Help/command/try_compile.rst#L120-L125

mwoehlke-kitware commented 2 months ago

I don't see how this has anything to do with the change cited? I'm not at all a Windows expert, but I do seem to recall running into files "locked" by the OS longer than you'd expect is a thing that can happen when compiling code. Maybe try asking on a more general CMake forum, especially one that gets more Windows users?

lucascolley commented 2 months ago

thanks for the quick response! will do

lucascolley commented 2 months ago

@eli-schwartz would you be able to tell me what call meson makes to cmake so that I can report the potential problem on their side?

eli-schwartz commented 2 months ago

I can look for this information later tonight. But off the top of my head I think we're supposed to log the exact call in the debug log...

lucascolley commented 1 month ago

I tried to build an sdist locally on windows to access a debug log here but I wasn't able to set up compilers correctly - building SciPy on windows can be a bit of a nightmare.

lucascolley commented 2 hours ago

This seems to have resolved itself?! Has anything changed on the meson side in relation to this as far as you know @eli-schwartz ?