imp: Support tsv and ssv prefixes (#2164)

reesmichael1 commented 4 months ago

With these changes, hledger now supports using the ssv: and tsv: prefixes to determine the file separator. For example:

$ cat basic_s
Date;Description;Id;Amount
12/11/2019;Foo;123;10.23
$ cat basic_s.rules
skip         1
fields       date, description, , amount
date-format  %d/%m/%Y
$ hledger print -f ssv:basic_s # stock hledger cannot parse the file
The hledger journal file "/home/michael/builds/hledger/ssv:basic_s" was not found.
Please create it first, eg with "hledger add" or a text editor.
Or, specify an existing journal file with -f or LEDGER_FILE.
$ stack exec -- hledger print -f ssv:basic_s # But this new build can!
2019-11-12 Foo
    expenses:unknown           10.23
    income:unknown            -10.23

Some notes from my end:

Since I was already poking around this area, I went ahead and created a new type for all of the storage formats supported by hledger, but if this is too wide-reaching a change, it would be straightforward to use something like Raw String, which would in turn be Raw "journal", Raw "timeclock", etc., for everything except the CSV/etc. files.
The comments in CsvReader.hs suggest that it should be possible to pass the rules file as an argument and have the data file be inferred. However, I wasn't able to figure out how to do this, either with stock hledger or with my build. This means I very well might have broken this feature. Could you please share how this is supposed to work so I can verify if it still functions?
I don't love adding ssvtest.sh, which is mostly a copy of csvtest.sh, but I wasn't immediately sure how else to avoid the csv:.*.csv pattern used in csvtest.sh.

simonmichael commented 4 months ago

Great, thank you!

I had a few thoughts but they're not too important; I'm inclined to merge this and maybe (or not) do a followup PR or tweaks at some point.

simonmichael commented 4 months ago

Our csv reader is really a Character Separated Values reader, it can handle any separator character. I guess these changes don't disrupt with that; for other characters the .csv extension or csv: prefix should be used, as before.

simonmichael commented 4 months ago

Clarified https://hledger.org/dev/hledger.html#data-formats a little.

reesmichael1 commented 4 months ago

Cool, thanks! I was also thinking about clarifying those docs in the shower this morning, but looks like you beat me to it :grinning:

simonmichael / hledger

imp: Support tsv and ssv prefixes (#2164) #2165