mollie / mollie-api-node

Official Mollie API client for Node
http://www.mollie.com
BSD 3-Clause "New" or "Revised" License
231 stars 63 forks source link

Throttle iteration API #283

Closed Pimm closed 2 years ago

Pimm commented 2 years ago

As described in this issue, developers should not request large sets of data from the Mollie API in most scenarios. This PR aims to discourage doing so, by introducing throttling.

If a developer (and thereby a merchant) requests a large set of data, the throttling will spread out the pressure on the Mollie API over a larger period of time. But additionally, it might send a signal to the developer, asking "hey, this thing you're doing, it's costly, is there no better way?"

(Closes #271)

Interface

In this implementation, the developer is in control of the throttling:

for await (let payment of mollieClient.payments.iterate({ valuesPerMinute: 5000 })) {
  […]
}

Developers are using this client voluntarily. They can opt-out of using it at any time, and just use curl to communicate with the Mollie API instead. This is why I feel "forcing" a certain amount of throttling is not productive. If a developer chooses to set the valuesPerMinute to Number.POSITIVE_INFINITY, that's a conscious decision on their part.

(valuesPerMinute is optional, making this change backward compatible.)

Better

A safer form of throttling ‒ in the sense that it's more difficult for developers to circumvent ‒ would be one that is applied on the Mollie API itself rather than in the client(s). When such throttling is introduced, it can be removed from the client.

Best

Neither spreading the requests over a longer period of time, nor asking the developer to find a way to make fewer request is an ideal solution. The real solution is making more efficient requests, by supporting filtering and searching in the Mollie API itself.

Once the Mollie API supports filtering payments by currency, for instance, we can expand the iteration API:

// Find the 10 latest euro payments.
const iterator = mollieClient.payments.iterate()
    .filter({ currency: 'EUR' })
    .take(10);
for await (let payment of iterator) {
    // Apply logic.
}

This would guarantee one single call to the Mollie API (v2/payments?currency=EUR&limit=10).