search

turt2live / matrix-appservice-instagram-media

Bridges Instagram posts to Matrix (chat not supported)
GNU General Public License v3.0
12 stars 0 forks source link

Need some kind of rate-limit queue for Instagram requests #1

Closed turt2live closed 7 years ago

turt2live commented 7 years ago

Requests can pile up quickly and cause the bridge to exceed the rate limit very quickly. Some kind of process queue should be put in place to ensure that we don't exceed the limit (and continue to make requests).

Things like user profile updates and media polling should be added to a queue that is processed as time goes on, trying to avoid the rate limit. If the rate limit is reached, a cooldown period before retrying the queue should be enabled.

lukebarnard1 commented 7 years ago

AFAIK matrix-js-sdk should look after rate limiting for you. You could also disable rate limiting from the AS registration file :innocent:

turt2live commented 7 years ago

This is more for the Instagram queuing. For users that don't authenticate with the bridge, we have to poll for updates. As user count increases, the number of requests also goes up. With the current code I was also hitting the sandbox rate limit (500/hr) easily.