UnicodeEncodeError in archive_util

[jaraco]

After fixing #709, a new error is revealed in the LC_ALL=C tests in CI.

______________________________ test_unicode_files ______________________________

tarfile_with_unicode = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/unicode-pkg-1.0.tar.gz'

tmpdir = local('/tmp/pytest-of-travis/pytest-0/test_unicode_files0')

    def test_unicode_files(tarfile_with_unicode, tmpdir):

        target = tmpdir / 'out'

>       archive_util.unpack_archive(tarfile_with_unicode, six.text_type(target))

setuptools/tests/test_archive_util.py:55: 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

setuptools/archive_util.py:53: in unpack_archive

    driver(filename, extract_dir, progress_filter)

setuptools/archive_util.py:169: in unpack_tarfile

    tarobj._extract_member(member, final_dst)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2108: in _extract_member

    self.makefile(tarinfo, targetpath)

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <tarfile.TarFile object at 0x7f86ff6c9a20>

tarinfo = <TarInfo './._testim\xe4ge.png' at 0x7f86ffefd8e0>

targetpath = '/tmp/pytest-of-travis/pytest-0/test_unicode_files0/out/./._testim\xe4ge.png'

    def makefile(self, tarinfo, targetpath):

        """Make a file called targetpath.

            """

        source = self.fileobj

        source.seek(tarinfo.offset_data)

>       with bltn_open(targetpath, "wb") as target:

E       UnicodeEncodeError: 'ascii' codec can't encode character '\xe4' in position 65: ordinal not in range(128)

../../../virtualenv/python3.5.2/lib/python3.5/tarfile.py:2148: UnicodeEncodeError

The issue lies in the "Ugh" code in archive_util.py.

[jaraco]

That didn't fix it. I'm giving up for now. If anyone else would like to take a stab at it, please feel free.

[pejorativefox]

Can confirm this, as it breaks many things upstream like zc.buildout. Clamping the setuptools version to 25.1.1 reverts to old behavior. To be honest though I can't really see anything in the diff that would do this........ Thanks for taking a stab at it @jaraco

[pganssle]

@jaraco Is this fixed by 6c6f4caa4bacce4b719b1e03fb02fdb857b1a135, or was that just a temporary workaround?

[mgorny]

Unless I'm doing something wrong, this is now XPASS-ing for me on 46.1.3, with PyPy3 7.3.0, CPython 3.{6.10,7.7,8.2}.

[mgorny]

Sorry, this → setuptools/tests/test_archive_util.py::test_unicode_files