Closed Bjwebb closed 2 years ago
Seems the python CSV model has problems with NULLs that make it impossible to use for UTF-16 encoded files:
http://docs.python.org/2/library/csv.html
The csv module doesn’t directly support reading and writing Unicode, but it is 8-bit-clean save for some problems with ASCII NUL characters. So you can write functions or classes that handle the encoding and decoding for you as long as you avoid encodings like UTF-16 that use NULs. UTF-8 is recommended.
A fix might be to use https://pypi.python.org/pypi/unicodecsv/
Looks like the CSV converter is not handling UTF-16 properly: