search

mithrandie / csvq

SQL-like query language for csv
https://mithrandie.github.io/csvq
MIT License
1.5k stars 65 forks source link

Ignore unequal length of columns #54

Closed muuuh closed 3 years ago

muuuh commented 3 years ago

Hello,

thank you for csvq!

Is it possible to add a feature to ignore unequal length of columns in a csv-file?

csvq -d ',' fields "test.csv"

results to

data parse error in file test.csv: line 2, column 0: wrong number of fields in line

It seems to be described here: https://stackoverflow.com/questions/34008600/handle-a-specific-error-in-golang

Thank you and best regards,

mithrandie commented 3 years ago

It's possible, but I think it is not a desirable feature because the behavior prevents users from noticing data corruption, and such corrupted data should be fixed by users before running queries. So I don't make it the default behavior, but if you want to be able to specify it as a command option, I'll consider it.

muuuh commented 3 years ago

I agree that this cannot be the default value and that the hints are helpful. However, it would be helpful as an optional setting to analyze certain data anyway. Maybe you could consider it. In any case, thank you.

mithrandie commented 3 years ago

The command option "--allow-uneven-fields" has been added in the version 1.15.1.