thomasdondorf / puppeteer-cluster

Puppeteer Pool, run a cluster of instances in parallel
MIT License
3.24k stars 310 forks source link

get progress #463

Open aymen3009 opened 2 years ago

aymen3009 commented 2 years ago

hello, I am using Puppeteer-cluster with Bullmq. and I was wondering if there is a way to get the progress of the queue so I can update the Bullmq job progress?

bibekgg commented 2 months ago

Old post but might be helpful for anyone who has similar request. What I have done is once we launch a cluster and assign a task for it, whenever we are executing the cluster use Promise.all and for the execute promise response update the count and the progress.

let done = 0;
let progress = 0;
const results = await Promise.all(
  arrayData.map(data =>
    cluster
      .execute(data)
      .then(res => {
        done++;
        progress = (done * 100) / arrayData.length;
        console.log('Progress -> ', progress.toFixed(2));
        return res;
      }),
  ),
);