pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.5k stars 1.19k forks source link

[BUG] setuptools.build_meta.build_wheel saves *.dist-info files in a duplicate sub-sub-directory, breaking wheels #4687

Closed davebelais closed 5 days ago

davebelais commented 6 days ago

setuptools version

75.2.0

Python version

3.8

OS

macOS

Additional environment information

No response

Description

Something went awry with building wheel metadata in 75.2.0...

To replicate, use the following:

import tempfile
from setuptools import build_meta

metadata_directory: str = tempfile.mkdtemp()
dist_directory: str = tempfile.mkdtemp()
build_meta.build_sdist(dist_directory)
build_meta.prepare_metadata_for_build_wheel(metadata_directory)
build_meta.build_wheel(
    dist_directory, metadata_directory=metadata_directory
)

Here's the output from the last line above when run w/ v75.1.0 (I am using this repo):

adding 'sob/__init__.py'
adding 'sob/abc.py'
adding 'sob/errors.py'
adding 'sob/hooks.py'
adding 'sob/meta.py'
adding 'sob/model.py'
adding 'sob/properties.py'
adding 'sob/py.typed'
adding 'sob/request.py'
adding 'sob/test.py'
adding 'sob/thesaurus.py'
adding 'sob/types.py'
adding 'sob/version.py'
adding 'sob/warnings.py'
adding 'sob/utilities/__init__.py'
adding 'sob/utilities/assertion.py'
adding 'sob/utilities/datetime.py'
adding 'sob/utilities/inspect.py'
adding 'sob/utilities/io.py'
adding 'sob/utilities/py.typed'
adding 'sob/utilities/string.py'
adding 'sob/utilities/types.py'
adding 'sob-1.50.10.dist-info/LICENSE'
adding 'sob-1.50.10.dist-info/METADATA'
adding 'sob-1.50.10.dist-info/WHEEL'
adding 'sob-1.50.10.dist-info/top_level.txt'
adding 'sob-1.50.10.dist-info/RECORD'

Here's the output from the last line of the same script w/ v75.2.0:

adding 'sob/__init__.py'
adding 'sob/abc.py'
adding 'sob/errors.py'
adding 'sob/hooks.py'
adding 'sob/meta.py'
adding 'sob/model.py'
adding 'sob/properties.py'
adding 'sob/py.typed'
adding 'sob/request.py'
adding 'sob/test.py'
adding 'sob/thesaurus.py'
adding 'sob/types.py'
adding 'sob/version.py'
adding 'sob/warnings.py'
adding 'sob/utilities/__init__.py'
adding 'sob/utilities/assertion.py'
adding 'sob/utilities/datetime.py'
adding 'sob/utilities/inspect.py'
adding 'sob/utilities/io.py'
adding 'sob/utilities/py.typed'
adding 'sob/utilities/string.py'
adding 'sob/utilities/types.py'
adding 'sob-1.50.10.dist-info/sob.egg-info/PKG-INFO'
adding 'sob-1.50.10.dist-info/sob.egg-info/SOURCES.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/dependency_links.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/requires.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/top_level.txt'
adding 'sob-1.50.10.dist-info/WHEEL'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/LICENSE'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/METADATA'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/top_level.txt'
adding 'sob-1.50.10.dist-info/RECORD'

...notice how the *.dist-info is double-nested when run in v75.2.0. This breaks the wheel, of course.

Expected behavior

I tried building a wheel using setuptools.build_meta.

How to Reproduce

import tempfile
from setuptools import build_meta

metadata_directory: str = tempfile.mkdtemp()
dist_directory: str = tempfile.mkdtemp()
build_meta.build_sdist(dist_directory)
build_meta.prepare_metadata_for_build_wheel(metadata_directory)
build_meta.build_wheel(
    dist_directory, metadata_directory=metadata_directory
)

Output

running sdist
running egg_info
writing sob.egg-info/PKG-INFO
writing dependency_links to sob.egg-info/dependency_links.txt
writing requirements to sob.egg-info/requires.txt
writing top-level names to sob.egg-info/top_level.txt
reading manifest file 'sob.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'sob.egg-info/SOURCES.txt'
running check
creating sob-1.50.10
creating sob-1.50.10/sob
creating sob-1.50.10/sob.egg-info
creating sob-1.50.10/sob/utilities
creating sob-1.50.10/tests
copying files to sob-1.50.10...
copying .scratch.py -> sob-1.50.10
copying LICENSE -> sob-1.50.10
copying README.md -> sob-1.50.10
copying pyproject.toml -> sob-1.50.10
copying setup.cfg -> sob-1.50.10
copying setup.py -> sob-1.50.10
copying sob/__init__.py -> sob-1.50.10/sob
copying sob/abc.py -> sob-1.50.10/sob
copying sob/errors.py -> sob-1.50.10/sob
copying sob/hooks.py -> sob-1.50.10/sob
copying sob/meta.py -> sob-1.50.10/sob
copying sob/model.py -> sob-1.50.10/sob
copying sob/properties.py -> sob-1.50.10/sob
copying sob/py.typed -> sob-1.50.10/sob
copying sob/request.py -> sob-1.50.10/sob
copying sob/test.py -> sob-1.50.10/sob
copying sob/thesaurus.py -> sob-1.50.10/sob
copying sob/types.py -> sob-1.50.10/sob
copying sob/version.py -> sob-1.50.10/sob
copying sob/warnings.py -> sob-1.50.10/sob
copying sob.egg-info/PKG-INFO -> sob-1.50.10/sob.egg-info
copying sob.egg-info/SOURCES.txt -> sob-1.50.10/sob.egg-info
copying sob.egg-info/dependency_links.txt -> sob-1.50.10/sob.egg-info
copying sob.egg-info/requires.txt -> sob-1.50.10/sob.egg-info
copying sob.egg-info/top_level.txt -> sob-1.50.10/sob.egg-info
copying sob/utilities/__init__.py -> sob-1.50.10/sob/utilities
copying sob/utilities/assertion.py -> sob-1.50.10/sob/utilities
copying sob/utilities/datetime.py -> sob-1.50.10/sob/utilities
copying sob/utilities/inspect.py -> sob-1.50.10/sob/utilities
copying sob/utilities/io.py -> sob-1.50.10/sob/utilities
copying sob/utilities/py.typed -> sob-1.50.10/sob/utilities
copying sob/utilities/string.py -> sob-1.50.10/sob/utilities
copying sob/utilities/types.py -> sob-1.50.10/sob/utilities
copying tests/test_meta_version.py -> sob-1.50.10/tests
copying tests/test_sob.py -> sob-1.50.10/tests
copying tests/test_utilities.py -> sob-1.50.10/tests
copying sob.egg-info/SOURCES.txt -> sob-1.50.10/sob.egg-info
Writing sob-1.50.10/setup.cfg
Creating tar archive
removing 'sob-1.50.10' (and everything under it)
running dist_info
creating /var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info
writing /var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/PKG-INFO
writing dependency_links to /var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/dependency_links.txt
writing requirements to /var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/requires.txt
writing top-level names to /var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/top_level.txt
writing manifest file '/var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/SOURCES.txt'
reading manifest file '/var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file '/var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob.egg-info/SOURCES.txt'
creating '/var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmpgs7wuup3/sob-1.50.10.dist-info'
running bdist_wheel
running build
running build_py
copying sob/hooks.py -> build/lib/sob
copying sob/version.py -> build/lib/sob
copying sob/thesaurus.py -> build/lib/sob
copying sob/properties.py -> build/lib/sob
copying sob/warnings.py -> build/lib/sob
copying sob/request.py -> build/lib/sob
copying sob/__init__.py -> build/lib/sob
copying sob/types.py -> build/lib/sob
copying sob/test.py -> build/lib/sob
copying sob/model.py -> build/lib/sob
copying sob/errors.py -> build/lib/sob
copying sob/abc.py -> build/lib/sob
copying sob/meta.py -> build/lib/sob
copying sob/utilities/io.py -> build/lib/sob/utilities
copying sob/utilities/__init__.py -> build/lib/sob/utilities
copying sob/utilities/types.py -> build/lib/sob/utilities
copying sob/utilities/assertion.py -> build/lib/sob/utilities
copying sob/utilities/inspect.py -> build/lib/sob/utilities
copying sob/utilities/string.py -> build/lib/sob/utilities
copying sob/utilities/datetime.py -> build/lib/sob/utilities
running egg_info
writing sob.egg-info/PKG-INFO
writing dependency_links to sob.egg-info/dependency_links.txt
writing requirements to sob.egg-info/requires.txt
writing top-level names to sob.egg-info/top_level.txt
reading manifest file 'sob.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'sob.egg-info/SOURCES.txt'
copying sob/py.typed -> build/lib/sob
copying sob/utilities/py.typed -> build/lib/sob/utilities
installing to build/bdist.macosx-14.2-arm64/wheel
running install
running install_lib
creating build/bdist.macosx-14.2-arm64/wheel
creating build/bdist.macosx-14.2-arm64/wheel/sob
copying build/lib/sob/hooks.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/version.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/thesaurus.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/properties.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/warnings.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/request.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/__init__.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/types.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/test.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/model.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
creating build/bdist.macosx-14.2-arm64/wheel/sob/utilities
copying build/lib/sob/utilities/io.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/__init__.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/types.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/assertion.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/inspect.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/py.typed -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/string.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/utilities/datetime.py -> build/bdist.macosx-14.2-arm64/wheel/./sob/utilities
copying build/lib/sob/py.typed -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/errors.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/abc.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
copying build/lib/sob/meta.py -> build/bdist.macosx-14.2-arm64/wheel/./sob
running install_egg_info
Copying sob.egg-info to build/bdist.macosx-14.2-arm64/wheel/./sob-1.50.10-py3.8.egg-info
running install_scripts
creating build/bdist.macosx-14.2-arm64/wheel/sob-1.50.10.dist-info/WHEEL
creating '/var/folders/bd/q6zm2xmd2472jdblpwpv_qvh0000gp/T/tmp2eq06ys1/.tmp-btl0dbrq/sob-1.50.10-py3-none-any.whl' and adding 'build/bdist.macosx-14.2-arm64/wheel' to it
adding 'sob/__init__.py'
adding 'sob/abc.py'
adding 'sob/errors.py'
adding 'sob/hooks.py'
adding 'sob/meta.py'
adding 'sob/model.py'
adding 'sob/properties.py'
adding 'sob/py.typed'
adding 'sob/request.py'
adding 'sob/test.py'
adding 'sob/thesaurus.py'
adding 'sob/types.py'
adding 'sob/version.py'
adding 'sob/warnings.py'
adding 'sob/utilities/__init__.py'
adding 'sob/utilities/assertion.py'
adding 'sob/utilities/datetime.py'
adding 'sob/utilities/inspect.py'
adding 'sob/utilities/io.py'
adding 'sob/utilities/py.typed'
adding 'sob/utilities/string.py'
adding 'sob/utilities/types.py'
adding 'sob-1.50.10.dist-info/sob.egg-info/PKG-INFO'
adding 'sob-1.50.10.dist-info/sob.egg-info/SOURCES.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/dependency_links.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/requires.txt'
adding 'sob-1.50.10.dist-info/sob.egg-info/top_level.txt'
adding 'sob-1.50.10.dist-info/WHEEL'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/LICENSE'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/METADATA'
adding 'sob-1.50.10.dist-info/sob-1.50.10.dist-info/top_level.txt'
adding 'sob-1.50.10.dist-info/RECORD'
removing build/bdist.macosx-14.2-arm64/wheel
abravalheri commented 5 days ago

Hi @davebelais thank you very much for reporting this.

Before we dive into investigation, could you please review the reproducer a little bit? If I understand the text of PEP 517 correctly the value passed to the build_wheel hook must be the same as returned by prepare_metadata_for_build_wheel, not the one passed in. So probably something like this:

import tempfile from setuptools
import build_meta

metadata_directory: str = tempfile.mkdtemp()
dist_directory: str = tempfile.mkdtemp() 
build_meta.build_sdist(dist_directory) 
metadata_directory = build_meta.prepare_metadata_for_build_wheel(metadata_directory)
build_meta.build_wheel(
    dist_directory,
    metadata_directory=metadata_directory, # UPDATE: this is also wrong see below.
)

Could you please check if the error still happens after reviewing the reproducer in the light of this nuance of PEP 517?

UPDATE: See below.

abravalheri commented 5 days ago

Actually, PEP 517 says:

build_wheel ... it should provide the path to the created .dist-info directory as the metadata_directory argument.

prepare_metadata_for_build_wheel Must create a .dist-info directory containing wheel metadata inside the specified metadata_directory (i.e., creates a directory like {metadata_directory}/{package}-{version}.dist-info/) ... This must return the basename (not the full path) of the .dist-info directory it creates, as a unicode string.

So I was also wrong, probably something like the following:

import os.path
import tempfile
from setuptools import build_meta

metadata_directory: str = tempfile.mkdtemp()
dist_directory: str = tempfile.mkdtemp()
build_meta.build_sdist(dist_directory)
dist_info = build_meta.prepare_metadata_for_build_wheel(metadata_directory)
build_meta.build_wheel(
    dist_directory, metadata_directory=os.path.join(metadata_directory, dist_info)
)

Could you please try something like that and let me know if the problem still persists?

davebelais commented 5 days ago

Thanks @abravalheri ! Yes, that works as expected. I should have thought to lookup the PEP :-).