search

brafdlog / caspion

Automated budget tracking from Israeli financial institutions
https://www.caspion.org
MIT License
195 stars 43 forks source link

Support hash changes and migrations #247

Closed baruchiro closed 3 years ago

baruchiro commented 3 years ago

Today, after israeli-bank-scrapers fixed the Leumi scraper, I ran the application again, and I got duplicated transactions in my Json and Google Sheets.

After checking it I found at least two changes:

  1. In my salary transactions, in the memo, there is one char changed: ' => `
  2. The date of the transaction changed from 2021-01-04T22:00:00.000Z to 2021-01-04T22:00:00.002Z (two milliseconds)

For a more robust hash, I guess we should ignore these kinds of differences.

I think we need to implement a function to compare hashes.

Now I'm thinking.... Why we have hash? Why are we not calculating it on demand? I can't remember.

baruchiro commented 3 years ago

We discussed about that before:

https://github.com/brafdlog/hiuvi/pull/112#discussion_r502927769

baruchiro commented 3 years ago

OK, first, the hash can't be recalculated, because it includes companyId but it doesn't added to the EnrichedTransaction:

https://github.com/brafdlog/hiuvi/blob/bb92beafc31a8685d583e353b1d45392f6a0ba97/src/backend/import/importTransactions.ts#L126-L136

Let's see which data is exported:

hash date chargedAmount description memo companyId accountNumber
sheets DD/MM/YY V V V V