search

thomasdondorf / puppeteer-cluster

Puppeteer Pool, run a cluster of instances in parallel
MIT License
3.21k stars 307 forks source link

some wonders about maxConcurrency #385

Open gxlfc opened 3 years ago

gxlfc commented 3 years ago

eg，i set maxConcurrency： 20

1、i wonder what maxConcurrency mean？，it mean 20 chromes？or 20 chrome tabs？ 2、i parallel request ，like 100 request，but i get error，so how to fix it~

{ Error: Protocol error (Network.getResponseBody): No resource with given identifier found at Promise (/home/work/nodejs/node_modules/puppeteer/lib/Connection.js:183:56) at new Promise () at CDPSession.send (/home/work/nodejs/node_modules/puppeteer/lib/Connection.js:182:12) at _contentPromise._bodyLoadedPromise.then (/home/work/nodejs/node_modules/puppeteer/lib/NetworkManager.js:618:45) at at process._tickCallback (internal/process/next_tick.js:188:7) -- ASYNC -- at Response. (/home/work/nodejs/node_modules/puppeteer/lib/helper.js:111:15) at Response.json (/home/work/nodejs/node_modules/puppeteer/lib/NetworkManager.js:639:32) at Response. (/home/work/nodejs/node_modules/puppeteer/lib/helper.js:112:23) at exports.Login (/home/work/nodejs/app/service/Login.js:149:34) at at process._tickCallback (internal/process/next_tick.js:188:7) message: 'Protocol error (Network.getResponseBody): No resource with given identifier found' }

drodriguezIE commented 3 years ago

@gxlfc maxConcurrency means the maximum number of parallel workers executing at any given time, for example if you ser maxConcurrency to 20 but you only have 3 tasks queued, it will open 3 browsers, but if you have 100 tasks queued, it will open 20 and as. soon as 1 or more of them is finished, it will open another browser until all 100 tasks are complete.