search

ome / omero-metadata

OMERO plugin for metadata manipulation https://www.openmicroscopy.org/omero/
GNU General Public License v2.0
7 stars 13 forks source link

Handle byte-order-mark (BOM) gracefully #39

Closed manics closed 4 years ago

manics commented 4 years ago

If a CSV starts with a byte-order-mark omero metadata populate fails with an unhelpful message along the lines of

$ omero metadata populate --file idr0069-screenA-annotation.csv Screen:2251

...lib/python2.7/site-packages/omero_metadata/populate.py", line 926, in preprocess_from_handle
    % first_row[header_index])
Exception: Empty column header in CSV: Plate

It should either print out a helpful error message (BOM found) or just handle it.

sbesson commented 4 years ago

Incidentally, came back to this as https://github.com/ome/omero-metadata/pull/40 now enforces the encoding of CSV files to support Unicode characters. Preliminary testing suggests that using utf-8-sig should be sufficient to also handle this issue and gracefully handle CSV files with BOM.

So far, this feels like the most convenient solution to this issue as it should equally handle CSV files with or without BOM. I will push extra commits to https://github.com/ome/omero-metadata/pull/40 before putting it up for review.