mechatroner / vscode_rainbow_csv

🌈Rainbow CSV - VS Code extension: Highlight CSV and TSV files in different rainbow colors to make them more readable
MIT License
425 stars 51 forks source link

Bad encoding leads to unaligned cells #103

Closed Chrismettal closed 1 year ago

Chrismettal commented 2 years ago

When the file you opened has chars that are badly encoded in it, the alignment will shift as if that char does not exist at all.

For example, the following string was saved with ANSI (Windows 1252) encoding:

This,is,a,header,with,°,bad,chars,°,sprinkled,throughout,
,,,,,,,,,,,
,,,,,,,,,,,

After opening it in VSCode, the default settings will try to open the file as UTF-8:

image

Obviously the real problem is opening the file with the wrong encoding, but trying to align the table regardless, because sometimes you just don't care if a single symbol is decoded correctly, the alignment will be off after each bad char:

image

While the problem does not start with the extension, maybe there is still a way to detect the bad char questionmark during counting for alignment as well.

mechatroner commented 2 years ago

Thanks for reporting. I agree that this is an issue, and this might actually be a special case of a more general problem, see #69

Chrismettal commented 2 years ago

You are right, I thought it was an issue of the <?> not being counted when padding other cells, but showing whitespace reveals that <?> just does not adhere to the fixed with font width like your referenced issue.

Well this is a difficult one to fix then.. Would need to add whitespace of a fixed width of the same size as the non-standard size character then..

image

image

mechatroner commented 1 year ago

I fixed #69 and it also resolves this issue, although alignment might still be a little bit "wobbly" which can be explained by font rendering issues: https://stackoverflow.com/a/7221188/2898283