Emergent failure in test_apply_pyproject_equivalent_to_setupcfg

[jaraco]

3 days ago, the head commit at 6067fa41a was passing tests, but when re-run here, two tests are failing (two forms of test_apply_pyproject_equivalent_to_setupcfg).

```python _ test_apply_pyproject_equivalent_to_setupcfg[https://github.com/aio-libs/aiohttp/raw/5e0e6b7080f2408d5f1dd544c0e1cf88378b7b10/setup.cfg] _ [gw1] linux -- Python 3.11.9 /home/runner/work/setuptools/setuptools/.tox/py/bin/python url = 'https://github.com/aio-libs/aiohttp/raw/5e0e6b7080f2408d5f1dd544c0e1cf88378b7b10/setup.cfg' monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f25c8dc2390> tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/popen-gw1/test_apply_pyproject_equivalen7') @pytest.mark.parametrize("url", urls_from_file(HERE / EXAMPLES_FILE)) @pytest.mark.filterwarnings("ignore") @pytest.mark.uses_network def test_apply_pyproject_equivalent_to_setupcfg(url, monkeypatch, tmp_path): monkeypatch.setattr(expand, "read_attr", Mock(return_value="0.0.1")) setupcfg_example = retrieve_file(url) pyproject_example = Path(tmp_path, "pyproject.toml") setupcfg_text = setupcfg_example.read_text(encoding="utf-8") toml_config = Translator().translate(setupcfg_text, "setup.cfg") pyproject_example.write_text(toml_config, encoding="utf-8") dist_toml = pyprojecttoml.apply_configuration(makedist(tmp_path), pyproject_example) dist_cfg = setupcfg.apply_configuration(makedist(tmp_path), setupcfg_example) _port_tests_require(dist_cfg) pkg_info_toml = core_metadata(dist_toml) pkg_info_cfg = core_metadata(dist_cfg) > assert pkg_info_toml == pkg_info_cfg E assert 'Metadata-Ver... "speedups"\n' == 'Metadata-Ver... "speedups"\n' E E Skipping 265 identical leading characters in diff, use -v to show E - =workflow%3ACI E + =workflow%%3ACI E ? + E Project-URL: Coverage: codecov, https://codecov.io/github/aio-libs/aiohttp E Project-URL: Docs: Changelog, https://docs.aiohttp.org/en/stable/changes.html... E E ...Full output truncated (30 lines hidden), use '-vv' to show setuptools/tests/config/test_apply_pyprojecttoml.py:57: AssertionError ----------------------------- Captured stdout call ----------------------------- Skipping https://github.com/aio-libs/aiohttp/raw/5e0e6b7080f2408d5f1dd544c0e1cf88378b7b10/setup.cfg (already exists: /home/runner/work/setuptools/setuptools/setuptools/tests/config/downloads/aio-libs_aiohttp5e0e6b7080f2408d5f1dd544c0e1cf88378b7b10_setup.cfg) ------------------------------ Captured log call ------------------------------- WARNING ini2toml.plugins:toml_incompatibilities.py:36 Sections 'flake8' ('setup.cfg') may be problematic. It might be the case TOML files are not supported by the relevant tools, or that 'ini2toml' just lacks a plugin for this kind of configuration. Please review the generated output and remove these sections if necessary. _ test_apply_pyproject_equivalent_to_setupcfg[https://github.com/sqlalchemy/sqlalchemy/raw/533f5718904b620be8d63f2474229945d6f8ba5d/setup.cfg] _ [gw1] linux -- Python 3.11.9 /home/runner/work/setuptools/setuptools/.tox/py/bin/python url = 'https://github.com/sqlalchemy/sqlalchemy/raw/533f5718904b620be8d63f2474229945d6f8ba5d/setup.cfg' monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7f25ca3b7c10> tmp_path = PosixPath('/tmp/pytest-of-runner/pytest-0/popen-gw1/test_apply_pyproject_equivalen10') @pytest.mark.parametrize("url", urls_from_file(HERE / EXAMPLES_FILE)) @pytest.mark.filterwarnings("ignore") @pytest.mark.uses_network def test_apply_pyproject_equivalent_to_setupcfg(url, monkeypatch, tmp_path): monkeypatch.setattr(expand, "read_attr", Mock(return_value="0.0.1")) setupcfg_example = retrieve_file(url) pyproject_example = Path(tmp_path, "pyproject.toml") setupcfg_text = setupcfg_example.read_text(encoding="utf-8") toml_config = Translator().translate(setupcfg_text, "setup.cfg") pyproject_example.write_text(toml_config, encoding="utf-8") > dist_toml = pyprojecttoml.apply_configuration(makedist(tmp_path), pyproject_example) setuptools/tests/config/test_apply_pyprojecttoml.py:51: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ setuptools/config/pyprojecttoml.py:68: in apply_configuration config = read_configuration(filepath, True, ignore_option_errors, dist) setuptools/config/pyprojecttoml.py:129: in read_configuration validate(subset, filepath) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ config = {'project': {'authors': [{'email': 'mike_mp@zzzcomputing.com', 'name': 'Mike Bayer'}], 'classifiers': ['Development St...cense-files': ['LICENSE'], 'package-dir': {'': 'lib'}, 'packages': {'find': {'namespaces': False, 'where': ['lib']}}}}} filepath = '/tmp/pytest-of-runner/pytest-0/popen-gw1/test_apply_pyproject_equivalen10/pyproject.toml' def validate(config: dict, filepath: StrPath) -> bool: from . import _validate_pyproject as validator trove_classifier = validator.FORMAT_FUNCTIONS.get("trove-classifier") if hasattr(trove_classifier, "_disable_download"): # Improve reproducibility by default. See issue 31 for validate-pyproject. trove_classifier._disable_download() # type: ignore try: return validator.validate(config) except validator.ValidationError as ex: summary = f"configuration error: {ex.summary}" if ex.name.strip("`") != "project": # Probably it is just a field missing/misnamed, not worthy the verbosity... _logger.debug(summary) _logger.debug(ex.details) error = f"invalid pyproject.toml config: {ex.name}." > raise ValueError(f"{error}\n{summary}") from None E ValueError: invalid pyproject.toml config: `project.optional-dependencies.postgresql_asyncpg[0]`. E configuration error: `project.optional-dependencies.postgresql_asyncpg[0]` must be pep508 setuptools/config/pyprojecttoml.py:57: ValueError ----------------------------- Captured stdout call ----------------------------- Skipping https://github.com/sqlalchemy/sqlalchemy/raw/533f5718904b620be8d63f2474229945d6f8ba5d/setup.cfg (already exists: /home/runner/work/setuptools/setuptools/setuptools/tests/config/downloads/sqlalchemy_sqlalchemy533f5718904b620be8d63f2474229945d6f8ba5d_setup.cfg) ------------------------------ Captured log call ------------------------------- WARNING ini2toml.plugins:toml_incompatibilities.py:36 Sections 'flake8' ('setup.cfg') may be problematic. It might be the case TOML files are not supported by the relevant tools, or that 'ini2toml' just lacks a plugin for this kind of configuration. Please review the generated output and remove these sections if necessary. ```

[jaraco]

Here's what changed between those builds (Python 3.11):

 setuptools main @ diff --unified orig.deps new.deps
--- orig.deps   2024-04-19 05:46:35
+++ new.deps    2024-04-19 05:45:27
@@ -10,10 +10,11 @@
 chardet==5.2.0
 charset-normalizer==3.3.2
 colorama==0.4.6
+ConfigUpdater==3.2
 coverage==7.4.4
 cryptography==42.0.5
 distlib==0.3.8
-docutils==0.20.1
+docutils==0.21.1
 domdf-python-tools==3.8.0.post2
 execnet==2.1.1
 filelock==3.13.4
@@ -26,11 +27,11 @@
 iniconfig==2.0.0
 jaraco.classes==3.4.0
 jaraco.context==5.3.0
-jaraco.develop==8.11.1
+jaraco.develop==8.13.0
 jaraco.env==1.0.0
 jaraco.envs==2.6.0
-jaraco.functools==4.0.0
-jaraco.packaging==9.5.0
+jaraco.functools==4.0.1
+jaraco.packaging==10.0.0
 jaraco.path==3.7.0
 jaraco.text==3.12.0
 jaraco.ui==2.3.0
@@ -69,11 +70,11 @@
 requests==2.31.0
 requests-file==2.0.0
 requests-toolbelt==1.0.0
-ruff==0.3.7
+ruff==0.4.0
 SecretStorage==3.3.3
-setuptools @ file:///home/runner/work/setuptools/setuptools/.tox/.tmp/package/1/setuptools-69.5.1.post20240416-0.editable-py3-none-any.whl#sha256=fadab4f1bd57c44a5fc7cc35cb87e69f42e632072f7bf894c2b33b9c844098c5
+setuptools @ file:///home/runner/work/setuptools/setuptools/.tox/.tmp/package/1/setuptools-69.5.1.post20240418-0.editable-py3-none-any.whl#sha256=4459986076a2dd5055c9f711ede2889dbad1d870c71fb2bd9c28b3d0bcdbecb0
 snowballstemmer==2.2.0
-Sphinx==7.2.6
+Sphinx==7.3.6
 sphinxcontrib-applehelp==1.0.8
 sphinxcontrib-devhelp==1.0.6
 sphinxcontrib-htmlhelp==2.0.5
@@ -85,10 +86,11 @@
 toml==0.10.2
 tomli==2.0.1
 tomli_w==1.0.0
+tomlkit==0.12.4
 tox==4.14.2
 typeguard==4.2.1
 typing_extensions==4.11.0
 urllib3==2.2.1
-virtualenv==20.25.1
-wheel==0.42.0
+virtualenv==20.25.3
+wheel==0.43.0
 zipp==3.18.1

[jaraco]

I'm able to replicate the same failure on my machine locally.

Uninstalling ConfigUpdater allows the tests to pass and reinstalling restores the failure.

[jaraco]

Some Python versions were released recently, so I thought that might be a factor, but both runs are Python 3.11.9.

I don't yet understand where ConfigUpdater comes from or why it's relevant. Searching the codebase, I don't find that string at all.

[jaraco]

I see now. Setuptools tests depend on ini2toml[lite], but late versions of jaraco.develop depend on ini2toml[full] and that's where ConfigUpdater gets pulled in and its presence violates the test assumptions. That dependency was added in https://github.com/jaraco/jaraco.develop/commit/52e113e30492eedc2927336f65cc4059f94e47ea as part of https://github.com/coherent-oss/roadmap/issues/8.

[jaraco]

I'm unsure the right course of action here. Ideally, the tests shouldn't be sensitive to the ini2toml flavor installed, but they are.

We could remove the setuptools[testing] dependency on jaraco.develop, which is required only to allow tools/finalize.py to be doctested (amounting to an import check at the moment).

For now I'll mark the test as xfail while we investigate/discuss.

[abravalheri]

Thanks @jaraco, for identifying that. I believe I can change that to stop failing (either by explicitly specifying the translator to use ConfigParser or by changing ini2toml directly).

The main difference is that ConfigUpdater is more similar to RawConfigParser than ConfigParser when it comes to parsing escaping sequences in the style %%.

Some time ago, I introduced a test case in setuptools inspired by the aiohttp's setup.cfg. That file contains such %%-style escaping sequence.