search

PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

[BUG] csvtokef fails with key error on file with 'UTF-8 with BOM' character encoding #536

Open Stephen-Veitch opened 1 month ago

Stephen-Veitch commented 1 month ago

Attempting to convert a csv of an offset table from csv to kef when the encoding of the source file is 'UTF-8 with BOM' fails with the following error:

Traceback (most recent call last):
  File "/Users/sveitch/anaconda3/envs/ph5/bin/csvtokef", line 11, in <module>
    load_entry_point('ph5', 'console_scripts', 'csvtokef')()
  File "/Users/sveitch/Documents/code/ph5/PH5/ph5/utilities/csvtokef.py", line 45, in main
    keffile.write("{0}\n".format(kef_dict['table']))

This operation will be successful with simple 'UTF-8' encoding. Excel will write csvs with BOM encoding under certain circumstances (I don't know when/why) so this is an issue that may be likely to come up for other users.

Stephen-Veitch commented 1 week ago

array_v0.utf8.csv array_v0.utf8bom.csv

Examples of array table csvs with UTF-8 and UTF-8 w/ BOM encodings. File contents are not particularly relevant as these are intended only as examples of working files with these encodings.