search

bartve / disconnect

An easy to use Node.js client with OAuth support to connect with the discogs.com API v2.0
MIT License
457 stars 79 forks source link

Rate Limiting Example in ReadMe? #30

Closed alololox closed 7 years ago

alololox commented 7 years ago

Hey, not so much an issue but I can't seem to get rate limiting to work and I notice that theres no documentation for it in the readme. I've tried to replicate from the queue.js file but I'm still getting

{ [DiscogsError: Too many requests] statusCode: 429, message: 'Too many requests' }

Any chance of including an example?

Thanks for the awesome tool :)

bartve commented 7 years ago

Sorry for the late reply. Never had issues with this myself, are you making calls to the Discogs API in a loop or something?

I thought there was no need for documentation as the rate limiting is done automatically. All calls that require rate limiting already implement it under the hood.

alololox commented 7 years ago

Yes, I'm making calls in a loop. I've got a JSON of albums that I'm searching discogs to grab an ID for, and then seaching masters by ID to pull image URLs.

So rate limiting has issues in loops? Any workaround suggestions for querying a large list of albums?

bartve commented 7 years ago

See https://www.discogs.com/developers/#page:home,header:home-rate-limiting The current request limit is 240 requests per minute per IP. Once reached, you'll have to wait until 1 minute after the first request for another batch of 240 requests. queue.js allows for a buffer of requests to be queued until the next minute interval. This is currently set to the default value of 20 for disconnect, .

I think I'm going to make this buffer length configurable through client.js like the other rate limiting settings to make it more flexible. Not sure what the memory usage effects are when setting a high value though. Until then you can just put a timer on your loop and after 240+20 requests have the loop timeout to 1 minute after the first request and proceed. It's basically what queue.js does when filling its buffer.