simonw / csvs-to-sqlite

Convert CSV files into a SQLite database
Apache License 2.0
878 stars 69 forks source link

CSVs with lines ending \r\n result in missing first column and shift in data columns #59

Open michalskop opened 4 years ago

michalskop commented 4 years ago

I the CSV file uses \r\n as it's row ending, the csvs-to-sqlite misses the first column (of data) and shifts the columns "to the left" (and the last column is therefore empty)

The \r\n ending is the default behaviour of Python's csv.writerow, see https://docs.python.org/3/library/csv.html#csv.Dialect.lineterminator

E.g.:

CSV:

id,value,number
1,3,2
2,1,7
3,4,1

results in SQLITE table:

id,value,number
3,2,
1,7,
4,1,