search

leolabs / you-need-a-parser

Convert CSV files to a format that can be imported in YNAB
https://ynap.leolabs.org
MIT License
33 stars 12 forks source link

Update existing Nordea format and add support for the new site #332

Closed mirague closed 4 years ago

mirague commented 4 years ago

Update: Re-created this issue in the bank2ynab repo as that seems more fitting (as the parsers come from there)

https://github.com/bank2ynab/bank2ynab/issues/339

leolabs commented 4 years ago

Thanks for the pointer! I'll update the parsers then. Could you tell me which encoding the CSV files have when exported from the site? Optimally, that would be UTF-8, but I know that a lot of banks use WIN-1252 instead which leads to some wonky characters when opened as UTF-8 :)

mirague commented 4 years ago

I've fixed the existing parser and added a new one with a PR in the bank2ynab repo -- do you automatically sync with those?

https://github.com/bank2ynab/bank2ynab/pull/340

The "old" format is UTF-8 The "new" format indicates UTF-8 with BOM

leolabs commented 4 years ago

Yeah, I sync with bank2ynab, but it's manually triggered. Just tell me when your PR has been merged and I'll trigger the sync :)

in the case of bank2ynab, YNAP tries to automatically detect the file's encoding. With UTF-8 (with and without BOM), this should be no problem.

mirague commented 4 years ago

Fantastic, cheers! Thanks for the snappy responses, glad to see you're active in this repo.

Does YNAP detect the bank purely by filename? This causes issues with bank2ynab: https://github.com/bank2ynab/bank2ynab/issues/339 where several configurations use the exact same name.

leolabs commented 4 years ago

It depends. In the case of the YNAP-native parsers, every bank script can decide to use its own methods to verify that the file can be parsed by it (see here, for example).

With bank2ynab parsers, I don't have that level of flexibility, so the matcher uses the filename and checks that the CSV has enough columns to satisfy the Input Columns config. In addition to that, it checks that the columns contain believable data. If you want to see all checks, you can see them in bank2ynab.ts.

Feel free to ask me if anything is still unclear :)

mirague commented 4 years ago

FYI: The PR just got merged :)

Seems your extra logic should suffice for this usecase!

leolabs commented 4 years ago

Cool! I've updated the configs in YNAP and it should support your new formats now. Could you try it out and check if it works? :)

mirague commented 4 years ago

Thanks, I've just tried it and these were my findings.

The old format - [SE Nordea - internetbanken.privat.nordea.se]

For the old format I notice it's not parsing amounts of 1.000,00 and above, it outputs NaN for those:

Input

Datum,Transaktion,Kategori,Belopp,Saldo
2020-05-05,Reservation Kortköp Spotify,,"-99,00",
2020-05-03,Reservation Kortköp HEMKOP,,"-34,95",
2020-05-05,Swish inbetalning JOHN DOE,,"3.000,00","8.778,76"
2020-05-05,Kortköp 200999 YOU NEED A BUDGET,,"-50,19","5.778,76"
2020-05-05,Autogiro K*SomeStore,,"-943,00","5.828,95"
2020-05-04,Kortköp 200999 APPLE COM BILL,,"-9,00","6.779,95"
2020-05-04,Kortköp 200999 STEAMGAMES COM,,"-165,00","6.788,95"
2020-05-01,Vardagspaket Månadspris kort,,"-12,00","6.953,95"
2020-04-30,Autogiro K*HEMKOP,,"-279,36","6.965,95"
2020-04-29,Autogiro K*DISCORD* N,,"-50,60","7.245,31"

Output

Category,Payee,Date,Inflow,Outflow
,Reservation Kortköp Spotify,05/05/2020,,99
,Reservation Kortköp HEMKOP,05/03/2020,,34.95
,Swish inbetalning JOHN DOE,05/05/2020,NaN,
,Kortköp 200999 YOU NEED A BUDGET,05/05/2020,,50.19
,Autogiro K*SomeStore,05/05/2020,,943
,Kortköp 200999 APPLE COM BILL,05/04/2020,,9
,Kortköp 200999 STEAMGAMES COM,05/04/2020,,165
,Vardagspaket Månadspris kort,05/01/2020,,12
,Autogiro K*HEMKOP,04/30/2020,,279.36
,Autogiro K*DISCORD* N,04/29/2020,,50.6

bank2ynab outputs correct numbers so perhaps a number conversion issue in YNAP?

The new format - [SE Nordea - netbank.nordea.se]

Nordea exports Invalid date for transactions that are reserved. bank2ynab skips these transactions but YNAP aborts the whole conversion saying there's an invalid date. Might be better if YNAP would proceed with the export, at most show a warning that some transactions could not be parsed.

Input

Bokföringsdag;Belopp;Avsändare;Mottagare;Namn;Rubrik;Saldo;Valuta
2020-05-05;3000,00;;NAID-SE-SEK-7001011234;;Swish inbetalning JOHN DOE;8778,76;SEK
2020-05-05;-50,19;NAID-SE-SEK-7001011234;;;Kortköp 200501 YOU NEED A BUDGET;5778,76;SEK
2020-05-05;-943,00;NAID-SE-SEK-7001011234;;;Autogiro K*SomeStore;5828,95;SEK
Invalid date;-99,00;NAID-SE-SEK-7001011234;;;Reservation Kortköp Spotify;;SEK
Invalid date;-34,95;NAID-SE-SEK-7001011234;;;Reservation Kortköp HEMKOP;;SEK
2020-05-04;-15,00;NAID-SE-SEK-7001011234;;;Kortköp 200428 PP*0000CODE;6771,95;SEK
2020-05-04;15,00;;NAID-SE-SEK-7001011234;;Återköp 200428 PP*0000CODE;6786,95;SEK

Output

None, an "Invalid date" error toast is shown.

leolabs commented 4 years ago

Hey, thanks for your feedback! I somehow didn't get a notification so I only saw it now. I'll see how I can fix these issues, but I already have some ideas in mind. I'll get back to you on Friday :)

leolabs commented 4 years ago

Sorry again for the delay. I've updated the algorithm, so invalid numbers should now be ignored instead of leading to an error :)

leolabs commented 4 years ago

I'll close this issue for now, but please feel free to re-open it if you have any other issues with your bank's statements.