search

oemof / oemof-tabular

Load oemof energy systems from tabular data sources.
https://oemof-tabular.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
7 stars 5 forks source link

Clean empty lines in csv before reading #140

Open nailend opened 1 year ago

nailend commented 1 year ago

A common reading error occurs when having multiple empty lines at the end of a csv file. tableschema.exceptions.CastError: Row length 0 doesn't match fields count 3

tableschema is expecting 3 columns but as an empty line doesn't have any column separators, there are none. This could be easily caught by cleaning the csv files from any empty lines while reading

nailend commented 1 year ago

its actually more complicated then I thought, as the csv would have to be modified itself, not sure if this is a good idea...

empty lines at the end could be detected, using raw_read()

raw_r = r.raw_read()
str(raw_r[-2:]) == "b'\\n\\n'"