How to add a version tag build when building with "build"

smartYSC commented 11 months ago

setuptools version

setuptools===69.0.0

Python version

Python 3.8.10

OS

Ubuntu 20.04 in WSL

Additional environment information

No response

Description

We are building test packages for branches and tag them as described here.

The build is executed with build===1.0.3 like this:

python -m build -C--build-option=egg_info -C--tag-build=.dev+foo

The equivalent command for setup.py would be something like this:

python setup.py egg_info --tag-build=.dev+foo sdist bdist_wheel

Building with setup.py works correctly in setuptools===68.2.2 AND 69.0.0. Building with build works in 68.2.2 but fails in 69.0.0. This is like due to the change of --build-option introduced in https://github.com/pypa/setuptools/pull/4079.

However, I cannot find a way to pass the egg_info --tag-build=.dev+foo into setuptools===69.0.0 via build.

I am not totally sure if this is a bug (because the change is documented) or if this is even an issue with setuptools or build. However, the interaction between both tools broke for us :-(

Is there a way to still tag a build through build when I cannot provide this as a build-option anymore?

Expected behavior

Building with setuptools via build should allow to tag builds.

How to Reproduce

Inside some project buildable with build (pyproject.toml and setup.py both work)

python -m venv venv
source venv/bin/activate
pip install build setuptools==68.2.2 wheel
python -m build -C--build-option=egg_info -C--tag-build=.dev+foo -n -> WORKS
pip install setuptools=69.0.0
python -m build -C--build-option=egg_info -C--tag-build=.dev+foo -n -> FAILS

Note that the example runs build with option -n to be able to change the version of setuptools by using a non-isolated build. Running without -n will always fail as it installs setuptools==69.0.0 in its own temporary virtual environment.

Output

python -m build -C--build-option=egg_info -C--tag-build=.dev+foo 
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
* Getting build dependencies for sdist...
/tmp/build-env-_gd_v1t2/lib/python3.8/site-packages/setuptools/dist.py:674: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated.
!!

        ********************************************************************************
        Please replace its usage with implicit namespaces (PEP 420).

        See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages for details.
        ********************************************************************************

!!
  ep.load()(self, ep.name, value)
running egg_info
<removed sensitive output>
* Building sdist...
/tmp/build-env-_gd_v1t2/lib/python3.8/site-packages/setuptools/dist.py:674: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated.
!!

        ********************************************************************************
        Please replace its usage with implicit namespaces (PEP 420).

        See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages for details.
        ********************************************************************************

!!
  ep.load()(self, ep.name, value)
running sdist
running egg_info
<removed sensitive output>
* Building wheel from sdist
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (setuptools >= 40.8.0, wheel)
* Getting build dependencies for wheel...
/tmp/build-env-8a3a_ks6/lib/python3.8/site-packages/setuptools/dist.py:674: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated.
!!

        ********************************************************************************
        Please replace its usage with implicit namespaces (PEP 420).

        See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages for details.
        ********************************************************************************

!!
  ep.load()(self, ep.name, value)
running egg_info
<removed sensitive output>
* Installing packages in isolated environment... (wheel)
* Building wheel...
/tmp/build-env-8a3a_ks6/lib/python3.8/site-packages/setuptools/dist.py:674: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated.
!!

        ********************************************************************************
        Please replace its usage with implicit namespaces (PEP 420).

        See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages for details.
        ********************************************************************************

!!
  ep.load()(self, ep.name, value)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: option --dist-dir not recognized

ERROR Backend subprocess exited when trying to invoke build_wheel

The error message printed is also somewhat misleading...

abravalheri commented 11 months ago

Hi @smartYSC, thank you very much for reporting this.

By all means please do consider support for config_settings in setuptools a feature that is highly unstable.

This is a complicated topic and I feel that we are kind of steering in circles: every time that we try to improve something that is suggested by someone we change something else that breaks things for other users...

Would something like the following work for you?

echo -e "[egg_info]\ntag_build=.dev+foo\n" > /tmp/build_opts.cfg
export DIST_EXTRA_CONFIG=/tmp/build_opts.cfg
python -m build

To me, the proper fix is to move https://github.com/pypa/setuptools/discussions/4083 ahead, but I have not got much feedback on that, nor people willing to champion the implementation.

smartYSC commented 11 months ago

Thank you for the immediate response!

I shortened this to:

echo -e "[egg_info]\ntag_build=.dev+foo\n" > /tmp/build_opts.cfg
DIST_EXTRA_CONFIG=/tmp/build_opts.cfg python -m build

And it works fine! I will change our build process accordingly.

Actually I cannot really help with the underlying problem you pointed out as I do not know (that) much about Python packaging.

Is your solution just a workaround that may break again in the (forseeable) future once #4083 is resolved? Or is this the "proper" way to provide build options?

Keep up the good work, I really appreciate that you are trying to solve the many issues outdated/deprecated Python packaging has!

abravalheri commented 11 months ago

I believe that right now the solution I pointed out is the most stable way of achieving the task you proposed. I don't think it will stop working even after #4083 is implemented.

It will stop working once we remove egg_info from setuptools, though (probably going to be replaced by dist_info because the word egg is a reference to an old unsupported format). But that is far ahead in the future.

abravalheri commented 11 months ago

Let's rename this topic and change categories so more people can find help in the future.

junshi356 commented 11 months ago

I also met this problem. For now, I plan to work around that by pinning setuptools<69.0.0. Our scenario is: I need to pack my package to a new "dev" version every time I run a script in a pipeline without manually updating metadata file, and I hope to only use pyproject.toml without introducing setup.py or setup.cfg to my project. Currently the only way we found is to use python build module with --build-option=egg_info and --build-option=--tag-build=dev123456 parameter, @abravalheri do you have suggestions how should we pack with dynamic versions after #4083?

abravalheri commented 11 months ago

Hi @junshi356, please see my previous comment about the most stable solution we have right now.

Note that the suggestion uses a throw-away file not included in the dist.

pypa / setuptools