If you run rapids-dependency-file-generator --clean with any file that has non-unicode characters in it, it will generate the following exception:
Traceback (most recent call last):
File "${VIRTUAL_ENV}/bin/rapids-dependency-file-generator", line 8, in <module>
sys.exit(main())
File "${VIRTUAL_ENV}/lib/python3.10/site-packages/rapids_dependency_file_generator/cli.py", line 103, in main
delete_existing_files(args.clean)
File "${VIRTUAL_ENV}/lib/python3.10/site-packages/rapids_dependency_file_generator/rapids_dependency_file_generator.py", line 40, in delete_existing_files
if HEADER in f.read():
File "${VIRTUAL_ENV}/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 6038: invalid start byte
There should be a try block around reading the file to ignore files that can not be decoded. Something like this fixed it for me:
def delete_existing_files(root="."):
"""Delete any files generated by this generator.
This function can be used to clean up a directory tree before generating a new set
of files from scratch.
Parameters
----------
root : str
The path to the root of the directory tree to search for files to delete.
"""
for dirpath, _, filenames in os.walk(root):
for fn in filter(
lambda fn: fn.endswith(".txt") or fn.endswith(".yaml"), filenames
):
try:
with open(file_path := os.path.join(dirpath, fn)) as f:
if HEADER in f.read():
os.remove(file_path)
except UnicodeDecodeError:
pass
If you run
rapids-dependency-file-generator --clean
with any file that has non-unicode characters in it, it will generate the following exception:There should be a try block around reading the file to ignore files that can not be decoded. Something like this fixed it for me: