personal-finance/actual-importer-code-samples/

[utterances-bot]

Actual Importer Code Samples — Red's Rants

So how minimal can we get our institution-specific code to be? Let’s look at best,average, and worst case examples.

https://reds-rants.netlify.app/personal-finance/actual-importer-code-samples/

[richban]

Here is a example of my statement.csv:

Type,Product,Started Date,Completed Date,Description,Amount,Fee,Currency,State,Balance
FEE,Current,2021-11-06 23:30:40,2021-11-06 23:30:40,Premium plan fee,-59.99,0.00,DKK,COMPLETED,1068.97
CARD_PAYMENT,Current,2021-10-31 15:26:36,2021-11-01 22:00:15,Iz *poulette Aps,-100.00,0.00,DKK,COMPLETED,1715.38
TOPUP,Current,2021-11-03 20:52:15,2021-11-03 20:52:18,Apple Pay Top-Up by *0393,2000.00,0.00,DKK,COMPLETED,2291.25
TRANSFER,Savings,2021-11-01 06:01:59,2021-11-01 06:01:59,To DKK Saving Account #2,1.00,0.00,DKK,COMPLETED,6.00

I am trying to implement my own importer for Revolut. The importer looks like this:

""" Revolut csv importer."""

from beancount_reds_importers.libreader import csvreader
from beancount_reds_importers.libtransactionbuilder import banking

class Importer(banking.Importer, csvreader.Importer):
    def custom_init(self):
        self.max_rounding_error = 0.14
        self.filename_identifier_substring = "revolut"

        # format specific to Revolut CSV
        self.date_format = "%m/%d/%Y"
        self.skip_head_rows = 1

        self.header_identifier = '"Transactions  for account ' + self.config.get(
            "main_account", "Assets:DK:Revolut:Checking"
        )
        # CSV column spec
        self.header_map = {
            "Type": "type",
            "Product": "product",
            "Started Date": "startedDate",
            "Completed Date": "completedDate",
            "Description": "description",
            "Amount": "amount",
            "Fee": "fees",
            "Currency": "currency",
            "State": "state",
            "Balance": "balance",
        }

        self.transaction_type_map = {
            "TRANSFER": "transfer",
            "CARD_PAYMENT": "expense",
            "TOPUP": "transfer",
            "EXCHANGE": "transfer",
            "ATM": "cash",
            "FEE": "fees",
        }

Running:

bean-identify config.py $file
bean-extract config.py -f $beancount_main $file

Produces no output or any error. Any thoughts what my importer is missing?

[redstreet]

Does bean-identify correctly identify it? If not, you have this line: self.filename_identifier_substring = "revolut"

Meaning, you'll have to rename your statement.csv to blahblah_revolutblah.csv.

[redstreet]

I took a quick look, and a few more things:

1. Set self.skip_head_rows = 0 as you don't have any lines to skip
2. self.skip_transaction_types = []: you will run into this once you get bean-identify working
3. self.header_identifier = 'Type,Product,Started Date,Completed Date,Description,Amount,Fee,Currency,State,Balance'
4. "Started Date": "date", You need at least one field to map to date
5. Your date format is 2021-11-06 23:30:40, which needs the time stripped. See prepare_raw_columns() and cleanup_date() within it in schwab_csv/init.py for an example of how to do this

[redstreet]

As an aside: when you have it all working, you can use the balance column to easily produce balance assertions. Example is in the schwab balances csv importer.

[richban]

Works like a charm! I'm still missing the balance assertions tho. Should I make the balance assertions based on the Started Date (max) and State == Completed? Since I can have PENDING transactions that don't have any balance yet:

CARD_PAYMENT | Current | 12/5/21 9:32 |   | Iz *Cooffe | -150 | 0 | DKK | PENDING |  
-- | -- | -- | -- | -- | -- | -- | -- | -- | --
CARD_PAYMENT | Current | 12/4/21 11:39 | 12/5/21 10:20 | Misc | -25 | 0 | DKK | COMPLETED | 1276.57
CARD_PAYMENT | Current | 12/4/21 13:03 | 12/5/21 10:20 | Iz * Coffee Aps | -79 | 0 | DKK | COMPLETED | 1197.57
CARD_PAYMENT | Current | 12/5/21 11:14 |   | Iz *Coffee | -30 | 0 | DKK | PENDING |  
TOPUP | Current | 12/5/21 11:47 | 12/5/21 11:48 | Top-Up by *0225 | 1680 | 0 | DKK | COMPLETED | 2877.57

[richban]

On a side note. Thanks for the amazing post! I'm will keep continue to work on it!

I am very fond of your site as well. Can you share some knowledge how did you turn your obsidian vault PKB into a website?

[redstreet]

Moved the discussion about balance assertions here.

[redstreet]

Aw, thank you for the kind words! :)

Reg the website, I plan to write an article on the entire setup at some point. But the core is simple: I didn't have to do much, since the "Digital garden Jekyll Template" does all the work once you point it at your Obsidian vault. A few plugins and such help get the flowcharts working. I actually now mostly use vim with vimwiki instead of Obsidian, unless I need flowcharts or visualizations.