facundoolano / google-play-scraper

Node.js scraper to get data from Google Play
MIT License
2.32k stars 629 forks source link

Throttling hangs on errors #520

Open protometa opened 2 years ago

protometa commented 2 years ago

Description:

The throttling code does not handle errors and hangs.

Example code:

See failing test in PR #521

Error message:

No error just hangs.

Giszmo commented 1 year ago

Describe the bug

When using the throttle option in the app call, things break in various ways when the app requested is not available.

Actual behavior

> for(var i=0;i<5;i++){gplay.app({appId:`does.not.exist.${i}`,throttle:3}).then(it=>{console.log('ok')}).catch(it=>{console.log('oops')})}
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 2121,
  [Symbol(trigger_async_id_symbol)]: 2120
}

... no further output. Ever. I suspect the throttled workers just died cause when processing my 4k appIds, it gets slower for every missing app and then stops.

Expected behavior

Same output as without throttle:

> for(var i=0;i<5;i++){gplay.app({appId:`does.not.exist.${i}`}).then(it=>{console.log('ok')}).catch(it=>{console.log('oops')})}
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 4092269,
  [Symbol(trigger_async_id_symbol)]: 4092268
}
> oops
oops
oops
oops
oops