search

nedbat / coveragepy

The code coverage tool for Python
https://coverage.readthedocs.io
Apache License 2.0
2.96k stars 428 forks source link

Internal use of unicode file paths on Python 2 #602

Open nedbat opened 7 years ago

nedbat commented 7 years ago

Originally reported by David MacIver (Bitbucket: davidmaciver, GitHub: Unknown)

Context: As discussed I am currently doing terrible things to Coverage.

As part of the Hypothesis build we run some of our tests on Python 2 under unicodenazi to check for implicit conversions between unicode and str and turn them into errors. This avoids finding encoding problems the hard way.

When I added coverage code I found a bunch of problems where it was mixing unicode and strings. The source (for me) seems to be that canonical_filename returns unicode, while Python file system operations expect paths to be a str.

In my case the fix was just that I encoded the output of canonical_filename with sys.getfilesystemencoding() . I don't know if that's actually sufficient, but it seems to work.

I can't say for sure if this is likely to be causing any problems. It might lead to weird errors when running in directories where there are non-ascii characters in the path? Or it might be a harmless false positive. I don't know, but I thought it worth mentioning.

nedbat commented 6 years ago

I'm not sure how to run with unicode-nazi and have it finish the test suite. For me, the whole test run abruptly ends with:

Traceback (most recent call last):
  File "/usr/local/pythonz/pythons/CPython-2.7.13/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/pythonz/pythons/CPython-2.7.13/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/unicodenazi.py", line 128, in <module>
    main()
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/unicodenazi.py", line 119, in main
    execfile(sys.argv[0], main_mod.__dict__)
  File "/usr/local/virtualenvs/coverage/bin/pytest", line 11, in <module>
    sys.exit(main())
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/config.py", line 58, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 139, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/main.py", line 133, in wrap_session
    exitstatus=session.exitstatus)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 250, in _wrapped_call
    wrap_controller.send(call_outcome)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/terminal.py", line 406, in pytest_sessionfinish
    outcome.get_result()
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 280, in get_result
    _reraise(*ex)  # noqa
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
    self.result = func()
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/flaky/flaky_pytest_plugin.py", line 214, in pytest_sessionfinish
    if hasattr(self.config, 'slaveoutput'):
  File "/Users/ned/coverage/trunk/.tox/py27/lib/python2.7/site-packages/unicodenazi.py", line 27, in warning_encode
    warnings.warn(UnicodeWarning('Implicit conversion of unicode to str'),
AttributeError: 'NoneType' object has no attribute 'warn'