ENOBUFS error after many requests at once (403 error after many frequent requests)

[ArtyCooL]

I'm trying to export reviews by looping through 2 arrays of data (10 games and 38 countries) and iTunes reviews pages. Something like this (I'm not good at coding):

for( var i = 0; i < g.length; i++) {
    for( var j = 0; j < c.length; j++) {
        for( var p = 0; p < 10; p++) {
            // Parse reviews
            })
            .then(console.log)
            .catch(console.log);
        };
    };
}

I'm getting this in the output file:

{ Error: connect ENOBUFS x.xx.xxx.xxx:xxx - Local (undefined:undefined)
    at Object._errnoException (util.js:992:11)
    at _exceptionWithHostPort (util.js:1014:20)
    at internalConnect (net.js:960:16)
    at defaultTriggerAsyncIdScope (internal/async_hooks.js:284:19)
    at GetAddrInfoReqWrap.emitLookup [as callback] (net.js:1106:9)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:97:10)
  code: 'ENOBUFS',
  errno: 'ENOBUFS',
  syscall: 'connect',
  address: 'x.xx.xxx.xxx',
  port: xxx }

I've searched for workaround and tried to delay each request with setTimeout (in many various ways) with no success.

I would like to know if it's possible to add throttle option like in google-play-scraper to avoid this error?

[facundoolano]

We don't currently support throttle in this library (mostly because up to now nobody had this issue), but it's a feature that could be added.

[ArtyCooL]

Here is the new error I'm getting when I'm trying to export reviews by each game (one by one in a short period of time). I guess it's all related to frequent requests to iTunes servers.

[Giszmo]

We don't currently support throttle in this library (mostly because up to now nobody had this issue), but it's a feature that could be added.

I would be very much interested in a throttle option. A team member sitting in the Philippines can't update our 311 iPhone app listings without dozens of errors.

If there is interest, I would offer a bounty of $100 in Bitcoin for this feature.

[facundoolano]

I don't have time or plans to work on this, but can review the code if someone else does. The google-play-scraper implementation can be used as a guide: https://github.com/facundoolano/google-play-scraper/blob/9473a885361dc4de4827bd709966064e09da09c2/lib/utils/request.js#L3-L15