search

codespell-project / codespell

check code for common misspellings
GNU General Public License v2.0
1.9k stars 466 forks source link

Chardet error should be caught and rethrown with helpful error message to user saying which file the problem occurred in #3519

Open corneliusroemer opened 2 months ago

corneliusroemer commented 2 months ago

I just ran codespell on all python peps with chardet -e. codespell crashed:

$ codespell --write-changes -i3 -C 5 -H -f -e --count -s --builtin clear,rare,names
Traceback (most recent call last):
  File "/Users/corneliusromer/.local/bin/codespell", line 8, in <module>
    sys.exit(_script_main())
             ^^^^^^^^^^^^^^
  File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 1121, in _script_main
    return main(*sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 1305, in main
    bad_count += parse_file(
                 ^^^^^^^^^^^
  File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 963, in parse_file
    lines, encoding = file_opener.open(filename)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 232, in open
    return self.open_with_chardet(filename)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 257, in open_with_chardet
    lines = f.readlines()
            ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/encodings/cp1254.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1349: character maps to <undefined>

The chardet error should be caught and rethrown with a better error message for the end user - I don't even know which file it is based on the error message as is!