[Bug]: GoCardless will implement strict ratelimiting

[tcrasset]

Verified issue does not already exist?

• [X] I have searched and found no existing issue

What happened?

I got the following mail from GoCardless. I think we should grab those headers from each response and keep count of where we're at, so that we can:

1. return a comprehensible error to the user when GoCardless limits are reached
2. filtering unecessary sync events by the user from the actual app,to make sure that "unnecessary" calls are not being made to the GoCardless API, because 4 syncs a day is not a lot...

Dear Customer,

We are reaching out to inform you of some important updates regarding our API that will enhance your experience and provide greater visibility into bank API rate limits.

Improvements in the API
=======================

As part of our effort to enhance transparency around API rate limits, we are implementing additional information in our API response - rate limit headers.

What are Rate Limit Headers?
============================

Rate limit headers are response HTTP header fields included in API responses that will inform you about your current usage and remaining API call limits. These headers will offer you clearer insight into your rate limits for all API requests.

The new headers will provide key details such as the total request limit, the number of requests remaining within a specified timeframe, and the time until the limits reset.
The introduction of new API rate limit headers will include the following response headers:

General Rate Limits (for all API requests):
===========================================

    HTTP_X_RATELIMIT_LIMIT: Indicates the maximum number of allowed requests within the defined time window.
    HTTP_X_RATELIMIT_REMAINING: Shows the number of remaining requests you can make in the current time window.
    HTTP_X_RATELIMIT_RESET: Provides the time remaining in the current window.

Account Success Request (for account details, transactions, and balances):

    HTTP_X_RATELIMIT_ACCOUNT_SUCCESS_LIMIT: Indicates the maximum number of allowed requests within the defined time window.
    HTTP_X_RATELIMIT_ACCOUNT_SUCCESS_REMAINING: Shows the number of remaining requests you can make in the current time window.
    HTTP_X_RATELIMIT_ACCOUNT_SUCCESS_RESET: Provides the time remaining in the current window.

How are these changes being made?
=================================

Starting from 19 August 2024, we will implement a limit of 10 API requests per day for each access scope (details, balances, transactions) on account id level as part of our ongoing efforts to enhance operational effectiveness and minimize potential issues for all our clients. This phased approach will allow us to optimize our services progressively.

We plan to establish a limit of four requests per day for each access scope in the next phase.

We will keep you informed and provide you with notice prior to the launch of the next phase, once we have finalized the timelines.

What does this mean for you?
============================

From 19 August 2024, there will be a 10 requests per day limit. However, we advise you to begin adjusting your applications to accommodate the upcoming four requests per day limit.

Please note that failed requests against details, balances, and transactions endpoints will not count against the bank API rate limits, but they will count towards the general[ Bank Account Data API rate limits](https://gocardless.cmail19.com/t/t-l-sljtyx-tyjdhjttdy-r/) Bank Account Data API rate limits.

As always, should you have any questions or require further assistance, please do not hesitate to reach out to our support team [here](https://gocardless.cmail19.com/t/t-l-sljtyx-tyjdhjttdy-y/).

For further details, please refer to our updated help article on bank rate limits [here](https://gocardless.cmail19.com/t/t-l-sljtyx-tyjdhjttdy-j/).

Thank you for your continued partnership.

Kind regards,

The GoCardless Bank Account Data team

What error did you receive?

No response

Where are you hosting Actual?

None

What browsers are you seeing the problem on?

No response

Operating System

None

[matt-fidd]

IThere was a bit of conversation around this in the discord earlier if you're interested!

1 - return a comprehensible error to the user when GoCardless limits are reached

I believe that this is possible in some capacity at the moment, but only once the rate limit has been reached. The message would have to be something simple like "the rate limit has been reached, please wait for it to reset" as we can not access the headers to give information on when the time window resets through nordigen-node as it is currently implemented.

I've raised nordigen/nordigen-node#62 in hopes that something can be done about this.

2 - filtering unecessary sync events by the user from the actual app,to make sure that "unnecessary" calls are not being made to the GoCardless API

This is tough if I'm understanding what you're proposing. How can we define an unnecessary sync event?

The easier route to this (this could have been what you meant too, I could have misunderstood!) is to optimise the way that actual performs bank syncs. Currently, any sync will use 3 api calls, which means that when the 4/day limit is imposed we're limiting users to 1 sync per day.

Looking at the code we could optimise the way we fetch relatively easily, I'm looking at a change to do it at the moment but I'll be moving house tomorrow so it's all a little hectic here as you can probably imagine.

My propsal is:

1. Drop the accountMetadata fetch (it's only used for IBAN and I can't see that it's actually used anywhere)
2. If it's an initial sync (no starting balance) then fetch the balance and calculate the starting balance
3. If it's not an initial sync, we can drop the balance query too

[matt-fidd]

Closing this as we've made the sync as lean as possible and errors are shown in the UI when the rate limit is exceeded.