Emergent test failure in test_license_is_a_string

[jaraco]

As reported in #4333, and first observed in this job a few days ago, one of the tests has started failing without any changes to the code.

_____________________ TestEggInfo.test_license_is_a_string _____________________
[gw0] linux -- Python 3.10.14 /home/runner/work/setuptools/setuptools/.tox/py/bin/python

self = <setuptools.tests.test_egg_info.TestEggInfo object at 0x7fd580ecfd30>
tmpdir_cwd = local('/home/runner/work/setuptools/setuptools')
env = '/tmp/setuptools-test.v_4kwomr'

    def test_license_is_a_string(self, tmpdir_cwd, env):
        setup_config = DALS(
            """
            [metadata]
            name=foo
            version=0.0.1
            license=file:MIT
            """
        )

        setup_script = DALS(
            """
            from setuptools import setup

            setup()
            """
        )

        path.build({
            'setup.py': setup_script,
            'setup.cfg': setup_config,
        })

        # This command should fail with a ValueError, but because it's
        # currently configured to use a subprocess, the actual traceback
        # object is lost and we need to parse it from stderr
        with pytest.raises(AssertionError) as exc:
            self._run_egg_info_command(tmpdir_cwd, env)

        # Hopefully this is not too fragile: the only argument to the
        # assertion error should be a traceback, ending with:
        #     ValueError: ....
        #
        #     assert not 1
        tb = exc.value.args[0].split('\n')
>       assert tb[-3].lstrip().startswith('ValueError')
E       AssertionError

/home/runner/work/setuptools/setuptools/setuptools/tests/test_egg_info.py:222: AssertionError

[jaraco]

Here's what tb looks like when the assertion fails:

(Pdb) !print('\n'.join(tb))
Traceback (most recent call last):
  File "/private/var/folders/f2/2plv6q2n7l932m2x004jlw340000gn/T/pytest-of-jaraco/pytest-31/test_license_is_a_string0/setup.py", line 3, in <module>
    setup()
  File "/Users/jaraco/code/pypa/setuptools/setuptools/__init__.py", line 104, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/code/pypa/setuptools/setuptools/_distutils/core.py", line 158, in setup
    dist.parse_config_files()
  File "/Users/jaraco/code/pypa/setuptools/.tox/py/lib/python3.12/site-packages/_virtualenv.py", line 22, in parse_config_files
    result = old_parse_config_files(self, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/code/pypa/setuptools/setuptools/dist.py", line 627, in parse_config_files
    setupcfg.parse_configuration(
  File "/Users/jaraco/code/pypa/setuptools/setuptools/config/setupcfg.py", line 190, in parse_configuration
    meta.parse()
  File "/Users/jaraco/code/pypa/setuptools/setuptools/config/setupcfg.py", line 501, in parse
    section_parser_method(section_options)
  File "/Users/jaraco/code/pypa/setuptools/setuptools/config/setupcfg.py", line 476, in parse_section
    self[name] = value
    ~~~~^^^^^^
  File "/Users/jaraco/code/pypa/setuptools/setuptools/config/setupcfg.py", line 294, in __setitem__
    parsed = self.parsers.get(option_name, lambda x: x)(value)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jaraco/code/pypa/setuptools/setuptools/config/setupcfg.py", line 366, in parser
    raise ValueError(
ValueError: Only strings are accepted for the license field, files are not accepted

Or more importantly:

(Pdb) pprint.pprint(tb[-3:])
['    raise ValueError(',
 'ValueError: Only strings are accepted for the license field, files are not '
 'accepted',
 '']

So it seems that something has changed about how exceptions are rendered or captured. Python 3.12.3 was released recently, so perhaps that's relevant.

[jaraco]

I also see that new releases of pytest were made on the 26th and 27th, so those could have changed how the exceptions were captured.

[jaraco]

Downgrading to pytest 8.0.2 works around the failure and 8.1.2 exhibits the failure.

[jaraco]

Using pytest 8.0.2, I was able to see what that traceback looked like with the earlier release:

 setuptools main @ .tox/py/bin/pip install -q 'pytest==8.0.2'
 setuptools main @ .tox/py/bin/python -m pdb -m pytest -p no:cov -p no:xdist -p no:perf -p no:checkdocs -k license_is_a_string -s
> /Users/jaraco/code/pypa/setuptools/.tox/py/lib/python3.12/site-packages/pytest/__main__.py(1)<module>()
-> """The pytest entry point."""
(Pdb) b setuptools/tests/test_egg_info.py:222
Breakpoint 1 at /Users/jaraco/code/pypa/setuptools/setuptools/tests/test_egg_info.py:222
(Pdb) c
============================================================== test session starts ===============================================================
platform darwin -- Python 3.12.3, pytest-8.0.2, pluggy-1.5.0
rootdir: /Users/jaraco/code/pypa/setuptools
configfile: pytest.ini
plugins: ruff-0.3.1, typeguard-4.2.1, mypy-0.10.3, home-0.5.1, enabler-3.1.1, timeout-2.3.1, xdist-3.6.1
collecting 397 items                                                                                                                             file: /Users/jaraco/code/pypa/setuptools/setuptools/tests/config/setupcfg_examples.txt
collected 1631 items / 1630 deselected / 1 selected                                                                                              

setuptools/tests/test_egg_info.py > /Users/jaraco/code/pypa/setuptools/setuptools/tests/test_egg_info.py(222)test_license_is_a_string()
-> assert tb[-3].lstrip().startswith('ValueError')
(Pdb) import pprint
(Pdb) pprint.pprint(tb[-4:])
['      raise ValueError(',
 '  ValueError: Only strings are accepted for the license field, files are not '
 'accepted',
 '  ',
 'assert not 1']