search

wadetb / Sublime-Text-Advanced-CSV

Efficiently format, edit, arrange, and evaluate cells in CSV files
169 stars 14 forks source link

Justify/Compact columns doubles quotes for empty fields #36

Closed christianhans closed 7 years ago

christianhans commented 7 years ago

Hi,

Using this CSV file:

"a","b"
"","456"

Then performing "CSV: Justify columns" results in this CSV file:

a   ,b  
"""",456

I.e. the two quotes for the empty field in line 2 are doubled. Same happens with Compact columns.

Best regards Christian

wadetb commented 7 years ago

This behavior is a poor result of the auto-quoting system. You can disable it by adding "auto_quote": false to your User settings, which is probably what you want for a document in which everything is already quoted.

Per RFC 4081 section 2.7, the "" in row 2 column 1 is treated as an escaped double quote. When justifying the document, it then decides that the escaped double quote needs to be quoted itself, per section 2.5. This leads to the odd sequence """" which is literally a CSV field containing one double-quote character.

However, reading the BNF notation in the spec more carefully, it's clear that ""=escaped-double-quote is only true when already inside quotes. So, I'm going to tighten up the parser which should improve behavior in the common case of an empty quoted field.

wadetb commented 7 years ago

Fixed in 1.1.5, thanks for these reports @christianhans. It's really helpful to have specific and easily-reproducible test cases to fix!

christianhans commented 7 years ago

Just tested 1.1.5 - the new behavior is perfect. This plugin really became indispensable for my daily workflow - keep up the great work!

wadetb commented 7 years ago

That's great to hear, thanks for the feedback!