Python 3.* & py.test: codecs.lookup... TypeError

[manuphatak]

Python Versions: 3.2, 3.3, and 3.4, on Linux.

is_ascii_encoding is setup to catch a LookupError, but I'm getting a TypeError instead:

.tox/py34/lib/python3.4/site-packages/click/_compat.py:29: TypeError
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

encoding = None

    def is_ascii_encoding(encoding):
        """Checks if a given encoding is ascii."""
        try:
>           return codecs.lookup(encoding).name == 'ascii'
E           TypeError: must be str, not None

This originates from calls to .echo and .secho. Encoding is returning None. Then the property .name is called on that None.

I've tried setting the environment variables, with no effect, as described here: http://click.pocoo.org/4/python3/#python3-surrogates

My app works just fine, but this is failing my tests.

Here's the monkeypatch hack I've been using for py.test (for anyone else who runs into this): edit: just to be clear, this fixes the problem:

@pytest.fixture(autouse=True)
def setup(monkeypatch):
    import codecs

    def is_ascii_encoding(encoding):
        """Checks if a given encoding is ascii."""
        try:

            return codecs.lookup(encoding).name == 'ascii'
        except (LookupError, TypeError):
            return False

    # monkeypatch.setattr('click.getchar', lambda: 'A')
    # monkeypatch.setattr('click.confirm', lambda _: True)
    monkeypatch.setattr('click._compat.is_ascii_encoding', is_ascii_encoding)

An example traceback (without the patch):

.tox/py34/lib/python3.4/site-packages/click/_compat.py:29: TypeError
______________________________ test_flash_message ______________________________

game = <Mock spec='Hangman' id='139711558786464'>
capsys = <_pytest.capture.CaptureFixture object at 0x7f1121d51b00>
presenter = <hangman.presenter.Presenter object at 0x7f1121d519b0>

    def test_flash_message(game, capsys, presenter):
        message = 'This test is a success'
>       presenter.write(game=game, message=message)

tests/test_presenter.py:225: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
build/lib/hangman/presenter.py:22: in wrapper
    return func(self, *args, **kwargs)
build/lib/hangman/presenter.py:129: in write
    self.click.secho('{0:45s}'.format(message), bold=True, fg='yellow')
.tox/py34/lib/python3.4/site-packages/click/termui.py:409: in secho
    return echo(style(text, **styles), file=file, nl=nl, err=err, color=color)
.tox/py34/lib/python3.4/site-packages/click/utils.py:257: in echo
    file = _default_text_stdout()
.tox/py34/lib/python3.4/site-packages/click/_compat.py:543: in func
    rv = wrapper_func()
.tox/py34/lib/python3.4/site-packages/click/_compat.py:356: in get_text_stdout
    return _force_correct_text_writer(sys.stdout, encoding, errors)
.tox/py34/lib/python3.4/site-packages/click/_compat.py:309: in _force_correct_text_writer
    if encoding is None and not _stream_is_misconfigured(text_writer):
.tox/py34/lib/python3.4/site-packages/click/_compat.py:258: in _stream_is_misconfigured
    return is_ascii_encoding(getattr(stream, 'encoding', None))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

encoding = None

    def is_ascii_encoding(encoding):
        """Checks if a given encoding is ascii."""
        try:
>           return codecs.lookup(encoding).name == 'ascii'
E           TypeError: must be str, not None

[hackebrot]

Hi @bionikspoon

I experienced a similar problem with the latest tox version and click on Python3. Maybe the solution we came up with is working for you too. Please have a look into the gist included in the PR. :smile:

https://github.com/audreyr/cookiecutter/pull/453

[mehaase]

I'm seeing the same issue with both the python -m unittest test runner and nosetests.

[mitsuhiko]

I pushed a change that makes a stream with encoding set to None become an ascii stream. This will not actually fix the issue but at least it will not fail with a type error. This probably happens because the environment is misconfigured.

[sjagoe]

I have seen with tox 2.0 (where it sanitizes the environment in which tests are run) many unicode Python 3 issues. Python 3 (at least on NixOS) becomes unable to look up the correct encoding and falls back to ascii, which click then rejects as broken.

I worked around this by adding passenv = LANG LOCALE_ARCHIVE to tox.ini. I am not sure if this would be any different on other distributions.

[untitaker]

This seems to be a fairly recent problem with tox, Werkzeug and other software hit similar problems.

[sjagoe]

Yes, I believe it was a new feature in tox 2.0, release a couple of weeks ago.

[hackebrot]

@sjagoe Yes, indeed.

Feel free to join the discussion on the pytest-dev mailinglist.

:information_source: https://mail.python.org/pipermail/pytest-dev/2015-May/002950.html

[homeworkprod]

This might be fixed as of tox 2.0.2, see https://bitbucket.org/hpk42/tox/issue/247/pass-through-lang-environment-variable-by