search

facebookarchive / codemod

Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source.
Apache License 2.0
4.11k stars 198 forks source link

Unicode crash should be handled gracefully #93

Closed adamjernst closed 6 years ago

adamjernst commented 7 years ago 
Traceback (most recent call last):
  File "/opt/homebrew/bin/codemod", line 9, in <module>
    load_entry_point('codemod==1.0.0', 'console_scripts', 'codemod')()
  File "/opt/homebrew/lib/python3.5/site-packages/codemod/base.py", line 1025, in main
    run_interactive(**options)
  File "/opt/homebrew/lib/python3.5/site-packages/codemod/base.py", line 167, in run_interactive
    for patch in suggestions:
  File "/opt/homebrew/lib/python3.5/site-packages/codemod/base.py", line 442, in generate_patches
    lines = list(open(path))
  File "/opt/homebrew/Cellar/python35/3.5.1/Frameworks/Python.framework/Versions/3.5/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 130: invalid continuation byte
adamjernst commented 7 years ago

Best to skip files with unicode errors, instead of failing entirely.