tabs in fields have adverse effects on columns boundaries and alignment

[slackwhere]

The following CSV data with embedded tabs should be enough to see the visual effect. I'm not sure of the best way to represent tabs in the output and maintain proper alignment. Perhaps provide options to convert them to some number of spaces or strip them? Stripping them would a helpful at the beginning/ending of fields but not the middle.

col1,col2,col3
dat1,dat2,dat3
dat1,dat2,dat3
dat1    ,dat2,dat3
dat1    ,dat2,dat3
dat1    ,dat2,dat3
dat1    ,dat2   ,dat3
dat1    ,dat2,dat3
dat1    ,dat2   ,dat3
dat1    ,dat2,dat3
dat1    ,dat2,dat3
dat1    ,dat2,dat3
dat1    ,dat2,dat3
dat1,   dat2,dat3
dat1,   dat2,dat3
dat1,dat2,dat3
dat1,da t2,dat3
dat1,dat2,dat3
dat1,dat2,dat3

[okbob]

should be fixed by commit bef5a268fc05898cdb6fee03380b0a2be3a2a822

please, check

[slackwhere]

I see that it works for your test4.csv file. But for some reason it's still not aligning on my test file.

On Sat, Dec 17, 2022 at 3:07 PM Pavel Stehule @.***> wrote:

should be fixed by commit bef5a26 https://github.com/okbob/pspg/commit/bef5a268fc05898cdb6fee03380b0a2be3a2a822

please, check

— Reply to this email directly, view it on GitHub https://github.com/okbob/pspg/issues/214#issuecomment-1356447908, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYYJU4LQR374J5EVQCE5XNTWNYMRZANCNFSM6AAAAAATBQJO44 . You are receiving this because you authored the thread.Message ID: @.***>

[okbob]

should be fixed now

[slackwhere]

Yes, that works 👍

Perhaps in the future you would consider a feature request for a setting to trim whitespace from the beginning/ending of fields? At least in the data sets that I've been provided it's usually an error that they exist.

Thank you for the quick turnaround.

Wayne

On Sun, Dec 18, 2022 at 2:06 PM Pavel Stehule @.***> wrote:

should be fixed now

— Reply to this email directly, view it on GitHub https://github.com/okbob/pspg/issues/214#issuecomment-1356855705, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYYJU4OQHYAUXIFEIHTG42TWN5OFFANCNFSM6AAAAAATBQJO44 . You are receiving this because you authored the thread.Message ID: @.***>

[okbob]

I am not sure what is correct design - I just reimplemented behaviour implemented in Postgres. I believe so CSV import in Postgres is very correct, and it doesn't throw tabs. The trimming is easy, when you don't support multilines, but pspg does it - and trimming multiline value requires little bit more complex algorithm (for robust implementation).

pspg does it with spaces, I'll check how can be done it with any white chars

[okbob]

last commit should to remove outer tabs too. Please, check

[slackwhere]

Yes, that works except when the field is quoted and contains a leading/trailing tab.

"data",data,data data,"data",data

Perhaps that was intentional though?

On Sun, Dec 18, 2022 at 3:05 PM Pavel Stehule @.***> wrote:

last commit should to remove outer tabs too. Please, check

— Reply to this email directly, view it on GitHub https://github.com/okbob/pspg/issues/214#issuecomment-1356865575, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYYJU4NZV5RXJR43SRKIUETWN5U7DANCNFSM6AAAAAATBQJO44 . You are receiving this because you authored the thread.Message ID: @.***>

[okbob]

I don't think so any can be cleaned inside quotes. This is syntax how to force to keep initial and trailing white spaces.