sindresorhus / p-queue

Promise queue with concurrency control
MIT License
3.39k stars 182 forks source link

Getting queued results back in order #167

Open pzich opened 2 years ago

pzich commented 2 years ago

Hello there!

I just started using PQueue in a few projects and it was super easy to integrate, great work!

One issue I'm running into is the ordering of the results. In my case I'm loading and processing chunks of data, and the amount of time that takes can vary, so sometimes later chunks are done processing sooner. I've made a contrived example to illustrate the issue.

import PQueue from 'https://esm.run/p-queue'

const pre = document.querySelector('pre');

const queue = new PQueue({ concurrency: 4 });

const loader = (v) => new Promise(resolve => {
  console.log('Loading', v);
  setTimeout(() => {
    console.log('Loaded', v);
    resolve(v);
  }, (Math.random() * 2 + 2) * 1000);
});

const printLn = (v) => {
  console.log('Printing', v);
  pre.innerText += `${v}\n`;
}

for (let i = 0; i < 20; i++) {
  queue.add(() => loader(i).then(printLn));
}
Screen Shot 2022-08-29 at 1 09 14 PM

What I'd like to be able to do is hold onto and wait on resolving these later queued results until the ones before have all resolved, then resolve these in order up to the latest resolved item in the queue.


I think this functionality is somewhat decoupled / orthogonal to PQueue itself, but I think it would be helpful to include as an option for the queue, as well as a standalone promise-fun package.

Apologies if this is already implemented here or as part of promise-fun. I looked through the docs here and all of the promise-fun packages that sounded relevant/similar, but didn't see anything quite like this.