fetchlogic / amz-products

Amazon Product Advertising API Wrapper
Apache License 2.0
15 stars 4 forks source link

Timing/Throttling the callback when requesting #3

Closed Hucho closed 8 years ago

Hucho commented 8 years ago

Hallo, me again...thanks for patching the locale feature. I did not test it yet but I will definitley test it soon... Before, one other problem: My code works basically like this:

function runQueries (QueryArray){

for (i = 0; 0 < QuerryArray.length; i++){

amazon.getItemsInBrowseNode({
  QuerryArray[i]
}, function(err, res){
  evaluate data and store it in a MongoDB
});
}
}

My QuerryArray holds a considerable amount of requests. When I start my function, the for-loop starts firing requests while the callback is behind...my app crashes somehow. Basically it does not but it finds a product with undefined parameters and that causes the crash. First thing I would like to do is to throttle resp. controll the request-firing. What would be your proposal? There are tools like node-rate-limiter but also js proprietary things like setIntervall ect...

Any help would be great!

Thanks!

Hucho

jwd2a commented 8 years ago

You might review the rate limits that Amazon imposes on the Product Advertising API: https://affiliate-program.amazon.com/gp/advertising/api/detail/faq.html

You might just try using setTimeout and clearTimeout to do this, that'd be the most simple solution, but it really depends on your implementation.

http://stackoverflow.com/questions/6967785/ways-to-throttle-ajax-requests

node-rate-limiter looks like it'd work well also: http://stackoverflow.com/questions/20253425/throttle-and-queue-up-api-requests-due-to-per-second-cap

Really doesn't matter, just make sure you're obeying that 1-call-per-second rule.