search

actualbudget / actual

A local-first personal finance app
https://actualbudget.org
MIT License
14.05k stars 1.12k forks source link

[Bug]: GoCardLess does not import all transactions for credit card (Bank of Norwegian, SE/DK) #1392

Closed leikoilja closed 1 year ago

leikoilja commented 1 year ago

Verified issue does not already exist?

What happened?

Similar to the issues stated in https://github.com/actualbudget/actual/issues/724#issuecomment-1576331381 by @Inrego, when enabling a credit card from the Bank of Norwegian (Swedish in my case) - the integration does not import all transaction but only:

For context, on July 24th this is the last transaction that actual sees under Available (first 10) transactions properties for new integration of institution in sortTransactions function logs:

    {
      transactionId: 'xxx',
      entryReference: 'xxx',
      bookingDate: '2023-06-15',
      valueDate: '2023-06-28',
      transactionAmount: { amount: '-15', currency: 'SEK' },
      currencyExchange: [
        {
          sourceCurrency: 'SEK',
          exchangeRate: '1.00',
          targetCurrency: 'SEK',
          quotationDate: '2023-06-14T00:00:00.0000000',
        },
      ],
      creditorName: 'HEMKOP HUVUSTA',
      remittanceInformationUnstructured: 'HEMKOP HUVUSTA SOLNA SE',
      additionalInformation: 'MISC FOOD STORES - DEFAULT',
      proprietaryBankTransactionCode:'Köp',
      internalTransactionId: 'xxx',
    },

there were a lot dozen more transaction made between June 15th and July 24th. In the example above: the bookingDate points to the date when the payment was made.

When i perform a raw API call to gocardless - they correctly return a full list of transactions, example of the transaction that i performed this month (18th of July):

        {
                "transactionId": "xxx",
                "bookingDate": "2023-07-18",
                "valueDate": "2023-08-28",
                "transactionAmount": {
                    "amount": "-250.43",
                    "currency": "SEK"
                },
                "currencyExchange": [
                    {
                        "sourceCurrency": "SEK",
                        "exchangeRate": "1.00",
                        "targetCurrency": "SEK",
                        "quotationDate": "2023-07-15T00:00:00.0000000"
                    }
                ],
                "creditorName": "OKQ8                     ",
                "remittanceInformationUnstructured": "OKQ8                      SOLLENTUNA    SE ",
                "additionalInformation": "Automated Fuel Dispensers           ",
                "proprietaryBankTransactionCode": "Köp",
                "internalTransactionId": "xxx"
            },

The problem with credit cards *at least the Swedish ones

After contacting GoCardLess support, this is the message they received from the bank: Here's what the bank says about the valueDates in the future:

For the credit card, you have interest free credit for up to 45 days. For Sweden we invoice from 15th of last month to 15th of next month, with an additional ~14 days to complete payment. So the 15th of August this customer will receive invoice for the transaction made 15th of July, with a due date (valueDate) of August 28th.

The verdict Following the explanation from the bank, the last transaction example (with creditorName "OKQ8") the valueDate is August 28th which is the last date when to pay for the purchase interest free. And since valueDate is in the future - Actual will not import this transaction and will simply ignore it.

IMHO, for this specific bank (since it's a credit card) - Actual must take bookingDate instead of valueDate. I am not sure the same logic applies for all credit cards, but might be something to look into.

What error did you receive?

No response

Where are you hosting Actual?

Locally via Yarn

What browsers are you seeing the problem on?

Other

Operating System

Mac OSX

leikoilja commented 1 year ago

(could it be that this issues belongs to https://github.com/actualbudget/actual-server)? 😅

kyrias commented 1 year ago

From the GoCardless documentatio:

valueDate | The Date at which assets become available to the account owner in case of a credit

This description seems... not very helpful for me, but I found this one instead from lewik.org which gives the definition in the context of PSD2:

a reference time used by a payment service provider for the calculation of interest on the funds debited from or credited to a payment account;

That to me makes it sound like we should not be using valueDate in Actual, or at the very least it should not be the primary value used. From the rest of the GoCardless documentation it sounds to me like if anything we should use bookingDate || bookingDateTime || valueDate || valueDateTime to make sure that we hopefully get some date.

kyrias commented 1 year ago

I guess the GoCardless description is half the sentence from the Berlin Group guideliness, the full one being:

Date and time at which assets become available to the account owner in case of a credit entry, or cease to be available to the account owner in case of a debit entry.

Based on this and reading some other documents it feels like valueDate either valueDate or bookingDate would be reasonable depending on the wanted semantics, though valueDate should be the date I would actually expect, i.e. the transaction date.

BUT, since GoCardless appears to be trying to follow the Berlin Group's NextGenPSD2 specification for their API then I would argue that the GoCardless integration with this bank is incorrect and should be changed.

@leikoilja Since the bank has said that the valueDate they send does not match the NextGenPSD2 definition of valueDate I feel like you should request that GoCardless change their integration with that bank to not forward the value under that name since the bank's use of the valueDate field does not match the GoCardless API documentation.

kyrias commented 1 year ago

As a 'local' workaround I think it would make sense for the server to be able to normalize transactions as well to be able to locally fix this for the affected bank.

kyrias commented 1 year ago

@leikoilja Could you post a (redacted is fine) instance of the Available account properties for new institution integration and Available (first 10) transactions properties for new integration of institution in calculateStartingBalance function log lines so I can try to add this normalization in the server?

Inrego commented 1 year ago

Want to add that I've been in contact with GoCardless about the issue. They've just forwarded it to Bank Norwegian and then nothing seems to happen

kyrias commented 1 year ago

Did you include the fact that this use of valueDate doesn't match what GoCardless says in their docs, and so they should really remove the field on their side if Bank Norwegian won't fix it within a reasonable timeframe?

Either way, if you could provide the data from the log lines I mentioned above which appear when linking an account with the bank then I can see about adding normalization on the Actual server side. :)

leikoilja commented 1 year ago

@kyrias, here is pretty formatted log spit from the actual server. Thanks for looking into it

  1. "Available (first 10) transactions properties for new integration of institution in sortTransactions function"
  2. "Available (first 10) transactions properties for new integration of institution in calculateStartingBalance function"
# Available (first 10) transactions properties for new integration of institution in sortTransactions function
top10Transactions = [
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-20",
        "valueDate": "2023-07-20",
        "transactionAmount": {"amount": "9906", "currency": "SEK"},
        "debtorName": "...",
        "remittanceInformationUnstructured": "Från Ilja",
        "proprietaryBankTransactionCode": "Betalning",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-17",
        "valueDate": "2023-07-17",
        "transactionAmount": {"amount": "2094", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-16T00:00:00.0000000",
            }
        ],
        "creditorName": "IKEA KUNGENS KURVA HFB RE",
        "remittanceInformationUnstructured": "IKEA KUNGENS KURVA HFB RE KUNGENS KURVA SE ",
        "additionalInformation": "Furniture, Home Furnishings, and Equ",
        "proprietaryBankTransactionCode": "CreditVoucher",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-17",
        "valueDate": "2023-07-17",
        "transactionAmount": {"amount": "799", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-15T00:00:00.0000000",
            }
        ],
        "creditorName": "IKEA KUNGENS KURVA HFB RE",
        "remittanceInformationUnstructured": "IKEA KUNGENS KURVA HFB RE KUNGENS KURVA SE ",
        "additionalInformation": "Furniture, Home Furnishings, and Equ",
        "proprietaryBankTransactionCode": "CreditVoucher",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569336900",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-179", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-14T00:00:00.0000000",
            }
        ],
        "creditorName": "AMAZONMKTPLC*JP6SC9HS5 WW",
        "remittanceInformationUnstructured": "AMAZONMKTPLC*JP6SC9HS5 WW WWW.AMAZON.SE LU",
        "additionalInformation": "MISC SPECIALTY RETAIL",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569372587",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-50", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-14T00:00:00.0000000",
            }
        ],
        "creditorName": "SUNDBYBERGSKLINIKEN",
        "remittanceInformationUnstructured": "SUNDBYBERGSKLINIKEN SUNDBYBERG SE",
        "additionalInformation": "MED/HEALTH SERVICES - DEF",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569307501",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-39", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-13T00:00:00.0000000",
            }
        ],
        "creditorName": "SL",
        "remittanceInformationUnstructured": "SL STOCKHOLM SE",
        "additionalInformation": "LOCAL COMMUTER TRANSPORT",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1568826151",
        "bookingDate": "2023-07-14",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-229.3", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-13T00:00:00.0000000",
            }
        ],
        "creditorName": "COOP VASTRA SKOGEN",
        "remittanceInformationUnstructured": "COOP VASTRA SKOGEN SOLNA SE",
        "additionalInformation": "GROCERY STORES/SUPERMARKETS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1567378218",
        "bookingDate": "2023-07-12",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-155", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-03T00:00:00.0000000",
            }
        ],
        "creditorName": "BROD & SALT BAGERI - S:T",
        "remittanceInformationUnstructured": "BROD & SALT BAGERI - S:T STOCKHOLM SE",
        "additionalInformation": "RESTAURANTS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1565327825",
        "bookingDate": "2023-07-09",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-1384", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-03T00:00:00.0000000",
            }
        ],
        "creditorName": "GlobalE /Vivobarefoot",
        "remittanceInformationUnstructured": "GlobalE /Vivobarefoot Amsterdam NL",
        "additionalInformation": "SHOE STORES",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1564783660",
        "bookingDate": "2023-07-08",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-94", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-06T00:00:00.0000000",
            }
        ],
        "creditorName": "BROD & SALT BAGERI -KARLB",
        "remittanceInformationUnstructured": "BROD & SALT BAGERI -KARLB STOCKHOLM SE",
        "additionalInformation": "RESTAURANTS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
        "booked": true,
    },
]
# Available (first 10) transactions properties for new integration of institution in calculateStartingBalance function {
balances = [
    {
        "balanceAmount": {"amount": "-4986.91", "currency": "SEK"},
        "balanceType": "closingBooked",
        "creditLimitIncluded": false,
        "referenceDate": "2023-07-31",
    },
    {
        "balanceAmount": {"amount": "-10425.58", "currency": "SEK"},
        "balanceType": "expected",
        "creditLimitIncluded": false,
        "referenceDate": "2023-07-31",
    },
    {
        "balanceAmount": {"amount": "89574.42", "currency": "SEK"},
        "balanceType": "interimAvailable",
        "creditLimitIncluded": false,
        "referenceDate": "2023-07-31",
    },
]

top10SortedTransactions = [
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-20",
        "valueDate": "2023-07-20",
        "transactionAmount": {"amount": "9906", "currency": "SEK"},
        "debtorName": "...",
        "remittanceInformationUnstructured": "Från Ilja",
        "proprietaryBankTransactionCode": "Betalning",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-17",
        "valueDate": "2023-07-17",
        "transactionAmount": {"amount": "2094", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-16T00:00:00.0000000",
            }
        ],
        "creditorName": "IKEA KUNGENS KURVA HFB RE",
        "remittanceInformationUnstructured": "IKEA KUNGENS KURVA HFB RE KUNGENS KURVA SE ",
        "additionalInformation": "Furniture, Home Furnishings, and Equ",
        "proprietaryBankTransactionCode": "CreditVoucher",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "bookingDate": "2023-07-17",
        "valueDate": "2023-07-17",
        "transactionAmount": {"amount": "799", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-15T00:00:00.0000000",
            }
        ],
        "creditorName": "IKEA KUNGENS KURVA HFB RE",
        "remittanceInformationUnstructured": "IKEA KUNGENS KURVA HFB RE KUNGENS KURVA SE ",
        "additionalInformation": "Furniture, Home Furnishings, and Equ",
        "proprietaryBankTransactionCode": "CreditVoucher",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569336900",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-179", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-14T00:00:00.0000000",
            }
        ],
        "creditorName": "AMAZONMKTPLC*JP6SC9HS5 WW",
        "remittanceInformationUnstructured": "AMAZONMKTPLC*JP6SC9HS5 WW WWW.AMAZON.SE LU",
        "additionalInformation": "MISC SPECIALTY RETAIL",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569372587",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-50", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-14T00:00:00.0000000",
            }
        ],
        "creditorName": "SUNDBYBERGSKLINIKEN",
        "remittanceInformationUnstructured": "SUNDBYBERGSKLINIKEN SUNDBYBERG SE",
        "additionalInformation": "MED/HEALTH SERVICES - DEF",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1569307501",
        "bookingDate": "2023-07-15",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-39", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-13T00:00:00.0000000",
            }
        ],
        "creditorName": "SL",
        "remittanceInformationUnstructured": "SL STOCKHOLM SE",
        "additionalInformation": "LOCAL COMMUTER TRANSPORT",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1568826151",
        "bookingDate": "2023-07-14",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-229.3", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-13T00:00:00.0000000",
            }
        ],
        "creditorName": "COOP VASTRA SKOGEN",
        "remittanceInformationUnstructured": "COOP VASTRA SKOGEN SOLNA SE",
        "additionalInformation": "GROCERY STORES/SUPERMARKETS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1567378218",
        "bookingDate": "2023-07-12",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-155", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-03T00:00:00.0000000",
            }
        ],
        "creditorName": "BROD & SALT BAGERI - S:T",
        "remittanceInformationUnstructured": "BROD & SALT BAGERI - S:T STOCKHOLM SE",
        "additionalInformation": "RESTAURANTS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1565327825",
        "bookingDate": "2023-07-09",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-1384", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-03T00:00:00.0000000",
            }
        ],
        "creditorName": "GlobalE /Vivobarefoot",
        "remittanceInformationUnstructured": "GlobalE /Vivobarefoot Amsterdam NL",
        "additionalInformation": "SHOE STORES",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
    {
        "transactionId": "uniqueId123...",
        "entryReference": "1564783660",
        "bookingDate": "2023-07-08",
        "valueDate": "2023-07-28",
        "transactionAmount": {"amount": "-94", "currency": "SEK"},
        "currencyExchange": [
            {
                "sourceCurrency": "SEK",
                "exchangeRate": "1.00",
                "targetCurrency": "SEK",
                "quotationDate": "2023-07-06T00:00:00.0000000",
            }
        ],
        "creditorName": "BROD & SALT BAGERI -KARLB",
        "remittanceInformationUnstructured": "BROD & SALT BAGERI -KARLB STOCKHOLM SE",
        "additionalInformation": "RESTAURANTS",
        "proprietaryBankTransactionCode": "Köp",
        "internalTransactionId": "anotherUniqueID123",
    },
]
kyrias commented 1 year ago

Great, thank you. Any chance you could get a Available account properties for new institution integration line as well? They're printed when you go through the linking process, though you don't have to finish any new linking on the Actual side once you're redirected back from the bank/GoCardless.

Also, of the three balances (closingBooked, expected, interimAvailable) which one corresponds to your current running balance that your bank reported at the time, including any uncleared transactions? Normally Actual would prefer closingBooked but since they have a strange use of valueDate I guess they might have a strange use of the closingBooked balance as well.

In particular what Actual uses this for is to calculate starting balances when setting up a new account. What we do is take the correct "current balance" and then we subtract all of the known transactions to then get the starting balance amount.

leikoilja commented 1 year ago

Absolutely, sorry, missed where to find it first.

  1. "Available account properties for new institution integration" 
    # Available account properties for new institution integration {
account = {
"resourceId": "af5c50fc-28d6-49dd-b950-xxx",
"bban": "10606024xxx",
"currency": "SEK",
"ownerName": "Ilja Leiko",
"name": "CreditCard",
"product": "CreditCard",
"cashAccountType": "CACC",
"status": "READY",
"id": "4eba882d-9c69-4761-a17a-xxxx",
"created": "2023-03-16T12:19:28.641021Z",
"last_accessed": "2023-07-31T17:34:41.801390Z",
"iban": "NO4210606xxxxxx",
"institution_id": "NORWEGIAN_SE_NORWNOK1",
"owner_name": "Ilja Leiko",
"institution": {
    "id": "NORWEGIAN_SE_NORWNOK1",
    "name": "Bank Norwegian",
    "bic": "NORWNOK1",
    "transaction_total_days": "730",
    "countries": ["SE"],
    "logo": "https://cdn.nordigen.com/ais/NORWEGIAN_FI_NORWNOK1.png",
    "supported_payments": {},
    "supported_features": ["pending_transactions", "private_accounts"],
    "identification_codes": [],
},
}
  2. From the 3 balances above:
    • expected stands for how much is actually been spent (the running balance)
    • interimAvailable stands for the remaining credit limit
    • closingBooked - not really sure what this represent, but i'd guess it might to do with something how credit card works and is representing some kind of balance that needs to be payed before the interest is applied?
kyrias commented 1 year ago

I'd appreciate it if one or both of you would be able to help test out my changes in actualbudget/actual-server#237.

Inrego commented 1 year ago

Here's the account properties for my danish branch account:

Available account properties for new institution integration {
  account: '{"resourceId":"c18a2583-bbac-4dbc-89bf-xxx","bban":"3006128xxxx","currency":"DKK","ownerName":"Redacted","name":"CreditCard","product":"CreditCard","cashAccountType":"CACC","status":"READY","id":"43b10c91-815e-4ec6-b8af-0335b5ddxxxx","created":"2023-01-16T09:24:54.699301Z","last_accessed":"2023-07-05T09:09:08.065921Z","iban":"NO973006128xxxx","institution_id":"NORWEGIAN_DK_NORWNOK1","owner_name":"Redacted","institution":{"id":"NORWEGIAN_DK_NORWNOK1","name":"Bank Norwegian","bic":"NORWNOK1","transaction_total_days":"730","countries":["DK"],"logo":"https://cdn.nordigen.com/ais/NORWEGIAN_FI_NORWNOK1.png","supported_payments":{},"supported_features":["pending_transactions","private_accounts"],"identification_codes":[]}}'
}
LudvigHz commented 1 year ago

Hi! I seem to have stumbled upon a similar issue with Danske Bank Norway. I have the same issue with pending transactions having a valueDate in the future (1st of next month usually). This also seems to go for booked transactions though. The valueDate is still the 1st the month after the transaction occurred. I'll attach some example transactions from GoCardless here:

Some transactions from June: (all are booked) Note that some of the transactions have a bookingDate in June and valueDate in early July. While others have a valueDate in August??? And why there is a transaction from July in there I don't know (probably because they sort by valueDate?

(Ignore my really funky transactions here, I use Curve which adds the CRV* to all transactions and Danske Bank seems to fuck up even more with the random \nEndToEndID: NOTPROVIDED in there.

            {
                "transactionId": "-----------",
                "entryReference": "4976006219",
                "bookingDate": "2023-06-16",
                "valueDate": "2023-08-01",
                "bookingDateTime": "2023-06-16T00:00:00Z",
                "valueDateTime": "2023-08-01T00:00:00Z",
                "transactionAmount": {
                    "amount": "-64.68",
                    "currency": "NOK"
                },
                "creditorName": "CRV*GITHUB",
                "remittanceInformationUnstructured": "CRV*GITHUB\nEndToEndID: NOTPROVIDED",
                "additionalInformation": "4976006219",
                "merchantCategoryCode": "7372",
                "internalTransactionId": "---------------"
            },
            {
                "transactionId": "-------------",
                "entryReference": "246E05912861110I",
                "bookingDate": "2023-07-14",
                "valueDate": "2023-08-01",
                "bookingDateTime": "2023-07-14T00:00:00Z",
                "valueDateTime": "2023-08-01T00:00:00Z",
                "transactionAmount": {
                    "amount": "0.72",
                    "currency": "NOK"
                },
                "remittanceInformationUnstructured": "Cashback denne måned\nEndToEndID: NOTPROVIDED",
                "additionalInformation": "246E05912861110I",
                "internalTransactionId": "----------------"
            },

            {
                "transactionId": "-------------------",
                "entryReference": "4967588441",
                "bookingDate": "2023-06-15",
                "valueDate": "2023-07-03",
                "bookingDateTime": "2023-06-15T00:00:00Z",
                "valueDateTime": "2023-07-03T00:00:00Z",
                "transactionAmount": {
                    "amount": "-384.00",
                    "currency": "NOK"
                },
                "creditorName": "CRV*NORGES TEKNISK-",
                "remittanceInformationUnstructured": "CRV*NORGES TEKNISK-\nEndToEndID: NOTPROVIDED",
                "additionalInformation": "4967588441",
                "merchantCategoryCode": "5399",
                "internalTransactionId": "-------------"
            },

And some transactions from August: (also booked)

{
                "transactionId": "---------------------",
                "entryReference": "5261232015",
                "bookingDate": "2023-08-09",
                "valueDate": "2023-09-01",
                "bookingDateTime": "2023-08-09T00:00:00Z",
                "valueDateTime": "2023-09-01T00:00:00Z",
                "transactionAmount": {
                    "amount": "-46.20",
                    "currency": "NOK"
                },
                "creditorName": "CRV*RUTERAPPEN",
                "remittanceInformationUnstructured": "CRV*RUTERAPPEN\nEndToEndID: NOTPROVIDED",
                "additionalInformation": "5261232015",
                "merchantCategoryCode": "4111",
                "internalTransactionId": "------------------------"
            },
            {
                "transactionId": "------------------------------",
                "entryReference": "5260179447",
                "bookingDate": "2023-08-09",
                "valueDate": "2023-09-01",
                "bookingDateTime": "2023-08-09T00:00:00Z",
                "valueDateTime": "2023-09-01T00:00:00Z",
                "transactionAmount": {
                    "amount": "-46.86",
                    "currency": "NOK"
                },
                "creditorName": "CRV*RUTERAPPEN",
                "remittanceInformationUnstructured": "CRV*RUTERAPPEN\nEndToEndID: NOTPROVIDED",
                "additionalInformation": "5260179447",
                "merchantCategoryCode": "4111",
                "internalTransactionId": "--------------------------"
            },

I skimmed through https://github.com/actualbudget/actual-server/pull/237 and looked at the fix there. But it seems you are still using the valueDate for booked transactions? Sadly it doesn't seem like there is much more information I can use here.

I could reach out to Danske Bank and ask them, but in my experience they are pretty useless. And it doesn't help that the Norway branch just got shut down and sold off (in effect in 2-3 years) so I highly doubt they'll care to do anything about it.

Is it an option to use the bookingDate for this bank in a similar solution like in https://github.com/actualbudget/actual-server/pull/237? If so, I'd be happy to submit a PR :) Since I'm not too familiar in banking and actual I thought I'd hear to see if anyone here has any thoughts on this.

kyrias commented 1 year ago

I skimmed through actualbudget/actual-server#237 and looked at the fix there. But it seems you are still using the valueDate for booked transactions? Sadly it doesn't seem like there is much more information I can use here.

No, bookingDate is used for booked transactions. valueDate is only used for pending transactions which at least for the Swedish branch apparently do not have a bookingDate. Please give an example of a pending transaction, including the expected actual date of the transaction.

LudvigHz commented 1 year ago

No, bookingDate is used for booked transactions. valueDate is only used for pending transactions which at least for the Swedish branch apparently do not have a bookingDate. Please give an example of a pending transaction.

Aah, I'm blind. I see it now. I haven't read trough the whole discussion in https://github.com/actualbudget/actual-server/pull/237 though, so I don't have the full context of the issue for Bank Norwegian as implemented there (with the pending transactions and whatnot). But at least in this issue the only problem mentioned is with the valueDate not being the expected value for credit cards? Which seems to align with the behavior I'm seeing with Danske Bank.

I don't know if Danske Bank shows the same behavior with pending transactions (I don't have any atm so I can't check. I'll make sure to validate that before I make a PR though).

I guess my question here is that it is correct for me to assume that the right path forward would be to implement a similar bank integration as in https://github.com/actualbudget/actual-server/pull/237, at least for the booked transactions (starting balance seems to be the same as well)?

kyrias commented 1 year ago

I guess my question here is that it is correct for me to assume that the right path forward would be to implement a similar bank integration as in actualbudget/actual-server#237, at least for the booked transactions (starting balance seems to be the same as well)?

Correct. :)