sindresorhus / p-queue

Promise queue with concurrency control
MIT License
3.39k stars 182 forks source link

p-queue hanging #163

Open leosbotelho opened 2 years ago

leosbotelho commented 2 years ago
[WTF Node?] open handles:
- Timers:
  - (58570 ~ 58 s) (anonymous) @ file:///.../node_modules/p-queue/dist/index.js:283
- Intervals:
  - (60000 ~ 60 s) (anonymous) @ file:///.../node_modules/p-queue/dist/index.js:322

dist/index.js:283 dist/index.js:322

First it hangs on dist/index.js:283; and then, if you let it run for a while, on dist/index.js:322 as well.

My code is analogous to:

(async () => {
  for (let i = 0;i < 4;i++) {
    await Q.add(() => sth())  
  }
  // await Q.onIdle()
  // console.log("this is reached (ofc)")
})()

say

// initialized outside the async arrow (lexical scope)
// also hangs when this is
//  initialized inside the async arrow ('async context')
const Q = new PQueue({
  concurrency: 100,
  intervalCap: 500,
  interval: 60 * 1000
})
$ nvm current
v16.15.0

$ lsb_release -d
Description:    Pop!_OS 20.04 LTS

re https://github.com/sindresorhus/p-throttle/issues/6#issuecomment-435339438

leosbotelho commented 2 years ago

This seems more dependable, in the foreseeable future:

import pLimit from "p-limit"
import pThrottle from "p-throttle"

// no feature parity
export default ({ concurrency, intervalCap, interval }) => {
  const limit = pLimit(concurrency)
  const throttle = pThrottle({
    limit: intervalCap,
    interval
  })
  return {
    add: th => throttle(() => limit(th))()
  }
}
dmgawel commented 1 year ago

I experienced the same issue when using larger interval, say 1 minute (interval: 60000). The whole program seem to hang for that minute.