pysetup create should not convert package_data to extra_files

[tiran]

BPO	13712
Nosy	@tiran, @merwok, @embray
Dependencies	bpo-11805: package_data only allows one glob per-package
Files	fix-pysetup-create-package_data.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields: ```python assignee = 'https://github.com/merwok' closed_at = created_at = labels = ['type-bug'] title = 'pysetup create should not convert package_data to extra_files' updated_at = user = 'https://github.com/tiran' ``` bugs.python.org fields: ```python activity = actor = 'eric.araujo' assignee = 'eric.araujo' closed = True closed_date = closer = 'eric.araujo' components = ['Distutils2'] creation = creator = 'christian.heimes' dependencies = ['11805'] files = ['24144'] hgrepos = [] issue_num = 13712 keywords = ['patch'] message_count = 6.0 messages = ['150610', '150618', '150653', '150669', '152666', '152677'] nosy_count = 5.0 nosy_names = ['christian.heimes', 'eric.araujo', 'alexis', 'python-dev', 'erik.bray'] pr_nums = [] priority = 'normal' resolution = 'fixed' stage = 'resolved' status = 'closed' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue13712' versions = ['3rd party', 'Python 3.3'] ```

[tiran]

As requested in http://bugs.python.org/issue13703#msg150609

./python Lib/test/regrtest.py test_packaging
[1/1] test_packaging
Warning -- threading._dangling was modified by test_packaging
Warning -- sysconfig._SCHEMES was modified by test_packaging
test test_packaging failed -- Traceback (most recent call last):
  File "/home/heimes/dev/python/randomhash/Lib/packaging/tests/test_create.py", line 168, in test_convert_setup_py_to_cfg
    """))
AssertionError: '[metadata]\nname = pyxfoil\nversion = 0.2\nsummary = Python bindings for the Xf [truncated]... != '[metadata]\nname = pyxfoil\nversion = 0.2\nsummary = Python bindings for the Xf [truncated]...
  [metadata]
  name = pyxfoil
  version = 0.2
  summary = Python bindings for the Xfoil engine
  download_url = UNKNOWN
  home_page = http://www.python-science.org/project/pyxfoil
  maintainer = André Espaze
  maintainer_email = andre.espaze@logilab.fr
  description = My super Death-scription
         |barbar is now on the public domain,
         |ho, baby !

  [files]
  packages = pyxfoil
      babar
      me
  modules = my_lib
      mymodule
  scripts = my_script
      bin/run
- extra_files = setup.py
+ extra_files = Martinique/Lamentin/dady
+     Martinique/Lamentin/mumy
+     Martinique/Lamentin/sys
+     Martinique/Lamentin/bro
+     setup.py
      README
-     pyxfoil/fengine.so
      Pom
      Flora
      Alexander
+     pyxfoil/fengine.so
-     Martinique/Lamentin/dady
-     Martinique/Lamentin/mumy
-     Martinique/Lamentin/sys
-     Martinique/Lamentin/bro

  resources =
      README.rst = {doc}
      pyxfoil.1 = {man}

1 test failed: test_packaging

[merwok]

Thanks, I will check this.

[merwok]

The bug is caused by code in packaging.create that iterates over a dict (package_data) to extend a list (extra_files). Instead of just calling sorted to make output deterministic, I’d prefer to fix that more serious behavior bug (see also bpo-13463, bpo-11805 and bpo-5302 for more !fun package_data bugs). Problem is that the setup.cfg syntax does not define how to give more than one value. If it’s judged acceptable to disallow paths with embedded spaces, we could do something like this:

[files] package_data = spam = first second third

Otherwise we’d need to use multiple lines (requested in bpo-5302):

[files] package_data = spam = first spam = second spam = third

We probably don’t want that. An intermediate idea:

[files] package_data = spam = first second third

Not sure this would be the nicest thing for people to write, and for us (me) to extend the setup.cfg parser for.

Anyway, attached patch fixes the code so that package_data in setup.py becomes package_data in setup.cfg and adapts the tests to check that, disabling multi-value package_data for now. I tested it with distutils2 and pypy, so it should fix the hash change in your clone.

[embray]

FWIW, I'm for the first option for specifying package_data:

[files] package_data = spam = first second third

I'm pretty sure this is how I ended up implementing it in d2to1, since I needed this functionality.

Theoretically spaces could be supported with an escape sequence, but I don't think that's worth complicating things for if package_data is deprecated anyways. I'm all for making it difficult for anyone trying to include filenames with spaces in their source code.

[1762cc99-3127-4a62-9baf-30c3d0f51ef7]

New changeset edb6f9fb54ac by Éric Araujo in branch 'default': Stop converting package_data to extra_files in pysetup create (bpo-13712). http://hg.python.org/cpython/rev/edb6f9fb54ac

[1762cc99-3127-4a62-9baf-30c3d0f51ef7]

New changeset 730c2e4aaf9c by Éric Araujo in branch 'default': Stop converting package_data to extra_files in pysetup create (bpo-13712). http://hg.python.org/distutils2/rev/730c2e4aaf9c

New changeset ea717d8e71d0 by Éric Araujo in branch 'python3': Merge fixes for bpo-13901, bpo-11805, bpo-13712 and other improvements http://hg.python.org/distutils2/rev/ea717d8e71d0