Coverage causes UnicodeDecodeError when running tox with Python 2 and 3

[GoogleCodeExporter]

What steps will reproduce the problem?
1. On Ubuntu 11.04, install Python 2.6, 2.7, 3.1 and 3.2
2. Use the stable codebase for nose
3. In the top directory, run these commands:
$ tox
...
[TOX] py32: commands succeeded
[TOX] py31: commands succeeded
[TOX] py27: commands succeeded
[TOX] py26: commands succeeded
$ tox
...
[TOX] ERROR: py32: commands failed
[TOX] ERROR: py31: commands failed
[TOX] py27: commands succeeded
[TOX] py26: commands succeeded

What is the expected output? What do you see instead?
The tests should consistently pass for python 3. Instead, we get this exception:
FAIL: Doctest: coverage_html.rst
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3.2/doctest.py", line 2113, in runTest
    raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for coverage_html.rst
  File "/home/garyd/devel/nose-stable/build/tests/functional_tests/doc_tests/test_coverage_html/coverage_html.rst", line 0

----------------------------------------------------------------------
File 
"/home/garyd/devel/nose-stable/build/tests/functional_tests/doc_tests/test_cover
age_html/coverage_html.rst", line 23, in coverage_html.rst
Failed example:
    run(argv=[__file__, '-v', '--with-coverage', '--cover-package=blah',
              '--cover-html', '--cover-html-dir=' + cover_html_dir,
              support, ],
        plugins=[Coverage()]) # doctest: +REPORT_NDIFF
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.2/doctest.py", line 1248, in __run
        compileflags, 1), test.globs)
      File "<doctest coverage_html.rst[6]>", line 4, in <module>
        plugins=[Coverage()]) # doctest: +REPORT_NDIFF
      File "/home/garyd/devel/nose-stable/build/tests/nose/plugins/plugintest.py", line 412, in run_buffered
        run(*arg, **kw)
      File "/home/garyd/devel/nose-stable/build/tests/nose/plugins/plugintest.py", line 401, in run
        run(*arg, **kw)
      File "/home/garyd/devel/nose-stable/build/tests/nose/core.py", line 284, in run
        return TestProgram(*arg, **kw).success
      File "/home/garyd/devel/nose-stable/build/tests/nose/core.py", line 118, in __init__
        **extra_args)
      File "/usr/lib/python3.2/unittest/main.py", line 124, in __init__
        self.runTests()
      File "/home/garyd/devel/nose-stable/build/tests/nose/core.py", line 197, in runTests
        result = self.testRunner.run(self.test)
      File "/home/garyd/devel/nose-stable/build/tests/nose/core.py", line 63, in run
        result.printErrors()
      File "/home/garyd/devel/nose-stable/build/tests/nose/result.py", line 110, in printErrors
        self.config.plugins.report(self.stream)
      File "/home/garyd/devel/nose-stable/build/tests/nose/plugins/manager.py", line 94, in __call__
        return self.call(*arg, **kw)
      File "/home/garyd/devel/nose-stable/build/tests/nose/plugins/manager.py", line 162, in simple
        result = meth(*arg, **kw)
      File "/home/garyd/devel/nose-stable/build/tests/nose/plugins/cover.py", line 166, in report
        self.coverInstance.stop()
      File "/home/garyd/devel/nose-stable/.tox/py32/lib/python3.2/site-packages/coverage/control.py", line 590, in html_report
        reporter.report(morfs, config=self.config)
      File "/home/garyd/devel/nose-stable/.tox/py32/lib/python3.2/site-packages/coverage/html.py", line 72, in report
        self.status.read(config.html_dir)
      File "/home/garyd/devel/nose-stable/.tox/py32/lib/python3.2/site-packages/coverage/html.py", line 246, in read
        status = pickle.load(open(status_file, "rb"))
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 0: ordinal not in range(128)

The root cause seems to be that the coverage_html.rst doctests in Python 2 
place their coverage output in the top-level functional_tests directory, 
whereas in Python 3 a new /build/tests/ directory is created for each run of 
tox (using "python setup.py build_tests").

So subsequent runs of tox will copy the Python 2 output from the top-level 
directory and use it for the python 3 test. Note that the pickled status.dat 
file used by coverage is incompatible between Python 2 and 3, and that the 
coverage files are not overridden.

The net effect of this is that the Python 3 doctests try to unpickle a Python 2 
coverage file, with the above result.

Original issue reported on code.google.com by gazi...@gmail.com on 14 Jul 2011 at 6:52

[GoogleCodeExporter]

The most straightforward workaround for this is to clean up the output from 
coverage as part of the doctest fixture. I have added a patch for this as a 
pull request to Jason's unstable branch on bitbucket.

I am not sure if there are other issues here that we need to worry about (eg. 
why doesn't coverage simply overwrite the existing output directory?)

Original comment by gazi...@gmail.com on 14 Jul 2011 at 7:36

[GoogleCodeExporter]

Thanks for the bug. Fixed in revision 490606463c0495ee0b3924491a1b8c454a4f890c

Original comment by kumar.mcmillan on 14 Jul 2011 at 4:22

• Changed state: Fixed