search

monarchmoney / mint-export-extension

Effortlessly export your Mint data to CSV.
MIT License
36 stars 3 forks source link

Adaptive rate limiting #48

Closed idpaterson closed 10 months ago

idpaterson commented 10 months ago

This pull request implements rate limiting that will adapt to latency in the Mint API. The extension will issue up to 20 requests per second with no more than 6 active at any given time. I recommend doing some testing to tweak those numbers, it may be possible to issue more requests but this default seemed respectful to the API.

Fixes #29. The current implementation of withRateLimit schedules all requests for an account to begin at 50ms intervals regardless of how long it takes the Mint API to respond. If the responses take more than a few milliseconds we end up with numerous concurrent requests. Once flooded like that, since the Mint API uses HTTP/2 the browser may not impose a maximum number of connections and the Mint API is likely to rate limit the requests.

When exporting daily account balances the requests to get the monthly history for each account are now rate limited as well. Otherwise the extension kicks off one request for every account all at essentially the same time.

This PR will make exporting a little slower when the Mint API is fast, but it increases the chances that export will continue to work when the API is bogged down.

stevenirby commented 10 months ago

@idpaterson thank you for this! I keep getting hit with the rate limit as well.