brafdlog / caspion

Automated budget tracking from Israeli financial institutions
https://www.caspion.org
MIT License
199 stars 43 forks source link

Error with Cal importer and missing transactions of the same charge with time proximity #317

Open Moshet opened 2 years ago

Moshet commented 2 years ago

Hi,

I've noticed that a few transactions are missing in my export file. After investigating it, I understood that the missing transactions are ones of same charged amount and that took place shortly after another transaction in the same business:

  1. 2 Bit transactions of the same amount, processed almost at the same time: only one of them is documented (happened on Amex, showed on Amex website as 2 separated transactions)
  2. Same for cash withdrawal of the same amount in a couple of minutes interval (happened on Isracard, showed on Isracard website as one double transaction)
  3. Same in Cal (happened in gas station in one visit)

Another thing, I have a recurring error with the Cal importer, it says "failed to fetch process date" but completes the export anyway, just without one of my credit cards.

Hope you can fix it soon:)

Thanks!

baruchiro commented 2 years ago

Hi, here is how we calculate the hash of transactions, and with this, we identify "duplicate" transactions:

https://github.com/brafdlog/caspion/blob/0eeaa39914313b9eeaaf53c1ed1e734141f57fce/src/backend/transactions/transactions.ts#L20

To support hash changes and failures, we process the hash with this function:

https://github.com/brafdlog/caspion/blob/0eeaa39914313b9eeaaf53c1ed1e734141f57fce/src/backend/transactions/transactions.ts#L6-L10

The question is, if the date of the two transactions is exactly the same?

Moshet commented 2 years ago

The date is the same and time difference between the transactions is few seconds-few minutes

brafdlog commented 2 years ago

In order to prevent duplicates, we use the data we have which is what @baruchiro showed. If all these parameters are equal between two transactions I don't see a way to distinguish that from a duplicate transaction. For cases where you know this happens, changing the memo/description of one transaction in the destination (google sheets/ynab) is a workaround. If you have a suggestion on how to implement this better I will be glad to hear

Moshet commented 2 years ago

Every transaction has its own unique ID right? Maybe different transaction IDs can help distinguishing between different transaction. Distinguishing between transactions only by date and charged amount is basically incorrect. If there's no such unique ID, then I suggest showing all transactions, even if they are suspiciously duplicated, because users would also like to know about duplicate transactions in order to contact the credit card issuer.

github-actions[bot] commented 3 days ago

Issue has been marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.