UnicodeDecodeError when generating a coverage report

[jaap3]

After upgrading to version 2.0 I suddenly get a UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 28856: invalid start byte.

This only happens when generating a report. So I added a print statement to read_template_source and was surprised that it is reading files outside of the template directory. In my case the offending file is a changelog tucked away in a static dir: static/tiny_mce/changelog.txt.

This seems to be caused by the additional file extensions that are checked. I temporarily worked around this by setting template_extensions = 'html'.

[nedbat]

Can you provide a complete error report? We should be more robust in the face of these sorts of things.

[jaap3]

Here's the full stracktrace, let me know if you need any more information:

$ coverage report
Traceback (most recent call last):
  File "/path/to/bin/coverage", line 8, in <module>
    sys.exit(main())
  File "/path/to/site-packages/coverage/cmdline.py", line 871, in main
    status = CoverageScript().command_line(argv)
  File "/path/to/site-packages/coverage/cmdline.py", line 621, in command_line
    **report_args
  File "/path/to/site-packages/coverage/control.py", line 913, in report
    return reporter.report(morfs, outfile=file)
  File "/path/to/site-packages/coverage/summary.py", line 45, in report
    for fr, analysis in get_analysis_to_report(self.coverage, morfs):
  File "/path/to/site-packages/coverage/report.py", line 70, in get_analysis_to_report
    analysis = coverage._analyze(fr)
  File "/path/to/site-packages/coverage/control.py", line 808, in _analyze
    return Analysis(data, it, self._file_mapper)
  File "/path/to/site-packages/coverage/results.py", line 20, in __init__
    self.statements = self.file_reporter.lines()
  File "/path/to/site-packages/django_coverage_plugin/plugin.py", line 318, in lines
    lexer = Lexer(self.source())
  File "/path/to/site-packages/django_coverage_plugin/plugin.py", line 308, in source
    self._source = read_template_source(self.filename)
  File "/path/to/site-packages/django_coverage_plugin/plugin.py", line 150, in read_template_source
    text = f.read().decode(charset)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 28856: invalid start byte

[nedbat]

I'm having trouble creating a test scenario that shows this (a side file that isn't meant to be rendered causing the UnicodeError). Can you give me a link to your real repo where the error is happening? Or more about your coverage and test settings?

[jaap3]

The project I've encountered this in is private. I'll see if I can create a reproducer.

[jaap3]

@nedbat here's a reproducer project:

foobar.tar.gz

It's been reduced to the bare minimum. To reproduce the issue install the requirements and run make.

[nedbat]

Thanks, this helped. A fix is in the works, but it's just to make it possible to suppress the error with [report] ignore_errors=True. You can also use [report] omit=staticfiles/* in your .coveragerc file to prevent it looking in that directory in the first place.

[nedbat]

Fixed in #83. Also, added a fix to coverage.py that would also have made this ignorable: https://github.com/nedbat/coveragepy/commit/f3a70c951e838e3cfab706b9a2d0459d783e5a4f