search

pypa / setuptools

Official project repository for the Setuptools build system
https://pypi.org/project/setuptools/
MIT License
2.52k stars 1.19k forks source link

Remove use of tag_build and tag_date in Setuptools #4530

Open jaraco opened 3 months ago

jaraco commented 3 months ago

For a long time, Setuptools has taken advantage of the --tag-build and --tag-date options of the build via setup.cfg to make builds outside of the release process produce version numbers like 72.1.0.post20240731, so they're not uploaded and are distinct from their released counterparts.

This approach has the unfortunate effect, however, of making setuptools unable to be built from the repo without incorporating that workaround.

For example:

 ~ 🐚 pip-run -v git+https://github.com/pypa/setuptools@v72.1.0 -- -c pass
Collecting git+https://github.com/pypa/setuptools@v72.1.0
  Cloning https://github.com/pypa/setuptools (to revision v72.1.0) to /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-req-build-b77hkm6t
  Running command git clone --filter=blob:none --quiet https://github.com/pypa/setuptools /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-req-build-b77hkm6t
  Running command git checkout -q 57ec6e527e73f04099836ff2773468a68a5d2fb9
  Resolved https://github.com/pypa/setuptools to commit 57ec6e527e73f04099836ff2773468a68a5d2fb9
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: setuptools
  Building wheel for setuptools (pyproject.toml) ... done
  Created wheel for setuptools: filename=setuptools-72.1.0.post20240731-py3-none-any.whl size=2338151 sha256=464d3ee80c96e970a03c7c1ea21101224cd1692f099176aa9faf05472ff51dc1
  Stored in directory: /private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pip-ephem-wheel-cache-ljvpiy2m/wheels/87/9b/20/f77304c66ef859fff1279ecc9d4096f80d3c2e0420de454433
Successfully built setuptools
Installing collected packages: setuptools
Successfully installed setuptools-72.1.0.post20240731

So even though the project is built from a tagged release, it produces the post release tag.

Moreover, there are better ways to handle this situation, such as setuptools_scm, which should be viable now that Setuptools has natural dependencies (vendored or not).

I'd like to take a two-stage approach:

  1. Remove the tags and let Setuptools build whatever version is in the file.
  2. Adopt setuptools_scm and rely on it to generate the post-release tag.

The first stage gets the project to a state where tagged releases can be built and reflect their number. It also means that untagged commits will be built with that same version, but that seems like a small price to pay.

Maybe these changes can both happen in the same release, but I want to be able to back out stage 2 if it causes undue disruption.

jaraco commented 3 months ago

Another reason to be cautious about adopting setuptools_scm is because it also implies the scm-based file finder, so adopting it will supersede (or augment) MANIFEST.in, and I'm unaware of a way to opt out of that behavior.

jaraco commented 3 months ago

Another reason to be cautious about adopting setuptools_scm is because it would need to be declared as a build-time dependency. It would be inappropriate to declare it as a runtime dependency, as that would cause that plugin (and its implicit behaviors) to be employed for every project building with Setuptools, which would be inappropriate. So it must be declared as a build-time dependency (and thus only used when building setuptools).

Since setuptools_scm is [built by setuptools](), adding even a build-time dependency on it will also trigger the bootstrap problem, so any solution probably needs to vendor setuptools_scm in such a way that it's only added to the sys.path when Setuptools is building itself.

abravalheri commented 3 months ago

I was thinking that there might be a different approach (more lightweight) for adopting setuptools-scm without creating the build-cycle, for example by running python -m setuptools_scm > some-file in the CI job... This way we benefit from the setuptools-scm, but when someone tries to build setuptools from the sdist they would simply fallback to the version in PKG-INFO (or other file generated by tools/finalize.py)...

Would this be an acceptable approach? (This would avoid problems with vendoring).

jaraco commented 3 months ago

I was thinking that there might be a different approach (more lightweight) for adopting setuptools-scm without creating the build-cycle, for example by running python -m setuptools_scm > some-file in the CI job... This way we benefit from the setuptools-scm, but when someone tries to build setuptools from the sdist they would simply fallback to the version in PKG-INFO (or other file generated by tools/finalize.py)...

Would this be an acceptable approach? (This would avoid problems with vendoring).

Interesting ideas. I am interested in solving more than just the issues with building sdist. Plus, I feel like we already have a methodology for vendoring, which we can apply to this approach. I also feel like we're getting rather close to a solution to the bootstrap problem, so maybe setuptools can simply declare the build-time dependency soon, so I don't want to invest too much in a new, separate methodology.

I very much like the concept of not relying on setuptools_scm in the sdist. In coherent-oss/system#12, I'm considering taking that concept to the extreme. In a different build system, I'm considering having the build-to-sdist step have all of the complicated logic with dependencies an all, but then generate an sdist reliant on flit-core with no dependencies. That approach doesn't generalize to something like setuptools, which performs important transformations (compiling) at the build stage, but it does illustrate a similar concept (materialize a lot of the dynamic behaviors in the generation of the sdist and don't rely on those dynamic behaviors when building the wheel). Setuptools could implement a similar approach with versions (materialize them in the sdist) and have a limited set of dependencies when compiling an sdist to a wheel (i.e. rely on PKG-INFO instead of setuptools_scm). This difference in dependencies could potentially be advertised in the PEP 517 get_requires_for_build_sdist vs. get_requires_for_build_wheel.

All of these ideas are rather new (incorporating setuptools_scm, alternate approaches), so I'm going to take some time and explore lots of options. I'd like to see what an approach with python -m setuptools_scm (or similar) might do. Please feel encouraged to draft an implementation and propose the change, especially if you feel that approach beats what's drafted in #4532 (which is broken atm).

RonnyPfannschmidt commented 3 months ago

I'd be happy to provide a opt out for file finders if setuptools provided a way to communicate with them, currently its simply not safely possible without gruesome hacks

jaraco commented 3 months ago

I haven't previously articulated my goals for this effort, so let me do so now:

nanonyme commented 1 month ago

I was thinking that there might be a different approach (more lightweight) for adopting setuptools-scm without creating the build-cycle, for example by running python -m setuptools_scm > some-file in the CI job... This way we benefit from the setuptools-scm, but when someone tries to build setuptools from the sdist they would simply fallback to the version in PKG-INFO (or other file generated by tools/finalize.py)...

Would this be an acceptable approach? (This would avoid problems with vendoring).

This doesn't sound fun to distros wanting to build from git that need to do bootstrap build setuptools->setuptools-scm without having setuptools priorhand.

nanonyme commented 1 month ago

It was not documented here but in my opinion (packager hat on) the biggest problem with tag_date is that setuptools built from git tag is not reproducible without hacking at setup.cfg.