Error when reading .coverage file

[adirizka7]

Error detected while processing function coverage#Show[9]..function coverage#Show[5]..<SNR>75_CoverageShow[9]..coverage#ShowCoverage[45]..4[13]..maktaba#python#Eval:
line    6:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/site-packages/coverage/data.py", line 293, in read_file
    self.read_fileobj(f)
  File "/usr/lib64/python3.7/site-packages/coverage/data.py", line 271, in read_fileobj
    data = self._read_raw_data(file_obj)
  File "/usr/lib64/python3.7/site-packages/coverage/data.py", line 311, in _read_raw_data
    go_away = file_obj.read(len(cls._GO_AWAY))
  File "/usr/lib64/python3.7/codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 99: invalid continuation byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/fedora/.vim/plugged/vim-coverage/python/vim_coverage.py", line 33, in GetCoveragePyLines
    cov.load()
  File "/usr/lib64/python3.7/site-packages/coverage/control.py", line 677, in load
    self.data_files.read(self.data)
  File "/usr/lib64/python3.7/site-packages/coverage/data.py", line 653, in read
    data.read_file(self.filename)
  File "/usr/lib64/python3.7/site-packages/coverage/data.py", line 297, in read_file
    filename, exc.__class__.__name__, exc,
coverage.misc.CoverageException: Couldn't read data from '/home/fedora/Desktop/GIT/tinydb/.coverage': UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 99: invalid continu
ation byte
E858: Eval did not return a valid python object

lib version

attrs==20.1.0
coverage==5.2.1
importlib-metadata==1.7.0
iniconfig==1.0.1
more-itertools==8.4.0
packaging==20.4
pluggy==0.13.1
py==1.9.0
pyparsing==2.4.7
pytest==6.0.1
pytest-cov==2.10.1
six==1.15.0
toml==0.10.1
zipp==3.1.0

[dbarnett]

Can you upload the .coverage file or provide details about where it came from? The error suggests the file is not a valid python coverage file.

[adirizka7]

Sure, here is the file https://drive.google.com/file/d/1SpHT06QoMvtz13NyQkM6UeOGoDSuI25W/view?usp=sharing . It works fine when I run coverage html tho.

[adirizka7]

Looks like i figured it out, previous .coverage file was produced by using --cov-report term . I updated the args to --cov-report html and the cov.load() works fine. Thanks!

[adirizka7]

oh no, it only works in separate python file

(venv) ➜  tinydb git:(master) ✗ python3
Python 3.6.11 (heads/master-dirty:86df475, Jul 31 2020, 00:00:00) 
[GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from coverage import Coverage
>>> cov = Coverage()
>>> cov.load()
>>> 

but error still occurs when running ':CoverageShow'

[dbarnett]

And the same error from :CoverageShow, or slightly different?

Might also be room to improve that error if we can narrow down a clear failure scenario and how to detect it in our code.

[adirizka7]

It was the same error. Turns out the error occurs because I forgot to activate virtualenv and I was using the older version of coverage in my os.

$ pip freeze | grep coverage
coverage==4.5.4

It's working fine when I use the newer 5.2.1 version.

can we catch this kind of error and rewrite it to a more informative error ?

[dbarnett]

Oof, yeah, I'd hope we could detect those kinds of trivial format compatibility issues, but that error from coverage is pretty nonspecific. Do you know if there's any helper to inspect the old format or good heuristic to check for files in the old format?

[adirizka7]

I've searched for the correct pytest version that produced the older format, but sadly, I couldn't find any. Best I can think of right now is to catch this error UnicodeDecodeError and rewrite it to something like coverage data is not valid, check your coverage version ....

[dbarnett]

:+1: I imagine providing a little context and trimming out some of the noise from that error would be a major improvement. In this case the one CoverageException line contains the essential info. Ideally the error would look something like ERROR: ShowCoverage failed to read coverage file: coverage.misc.CoverageException: Couldn't read data from '/home/fedora/Desktop/GIT/tinydb/.coverage': UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf0 in position 99: invalid continuation byte.

That said, could take me quite some time to get around to submitting a fix. Contributions would be very welcome!

[adirizka7]

Nice, I'll find some time to work on this one if that's ok

[dbarnett]

Yes, that would be great!

You can include the coverage version from coverage.__version__ in the error message and just say it expected a coverage file compatible with that, check for mixed installations, and regenerate .coverage if needed. And then try to keep the original error message included in some form in case there's another cause for errors like a bug in the coverage library that the user would need to troubleshoot.

[eprigorodov]

Having the same issue. The traceback tells that plugin tries to read file .coverage as UTF-8 text, whereas it is actually an SQLite database:

$ file .coverage 
.coverage: SQLite 3.x database, last written using SQLite version 3022000

That is the case since coveragepy==5.0a2. The code supporting JSON format has then been removed in coveragepy==5.0a6.

Just tested that reverting to coveragepy==4.5.4 fixes the issue.

[dbarnett]

Oh, so probably it was the same unsupported SQLite issue all along! That sounds like we could catch and give a much more user-friendly message for this case, when it hits a UnicodeDecodeError trying to decode a file that's actually SQLite.