search

thomasdondorf / puppeteer-cluster

Puppeteer Pool, run a cluster of instances in parallel
MIT License
3.24k stars 310 forks source link

Clear up Concurrency wording incorrect usage #512

Closed j-mendez closed 1 year ago

j-mendez commented 1 year ago

This package states concurrency when this is not the same thing as parallel. Nodejs does not have the ability to run concurrent without using the expensive child processes that require the heavy runtime per instance. When using fork on nodejs via web server is also not a good idea unless the forks are spawned prior/limited very heavy operation.

j-mendez commented 1 year ago

The only operations that run concurrently in node are IO operations. You also cannot control the IO without special configuration. Example certain IO APIs outside ecma may use libuv which may or not perform the action the operation concurrent/spawn thread pools etc.

tamusjroyce commented 1 year ago

concurrent seems to be correct usage

Although node runs single threaded here (deno, with less expensive concurrency, may change that, once puppeteer is supported: https://github.com/denoland/deno/issues/16298). Chromium based browsers run each tab as its own process.

WIth the example, we technically have 3 concurrent processes running. Server and 2 Client tabs running on the server.

j-mendez commented 1 year ago

concurrent seems to be correct usage

Although node runs single threaded here (deno, with less expensive concurrency, may change that, once puppeteer is supported: denoland/deno#16298). Chromium based browsers run each tab as its own process.

WIth the example, we technically have 3 concurrent processes running. Server and 2 Client tabs running on the server.

I should have pin pointed what exact concurrency mentions. In the docs it states you can add a custom ConcurrencyImplementation in js which is not the same. From the nodejs aspect it does not matter if there is 100 browsers running. The execution is not happening concurrent to connect and drive the pages. Using a word that has a bigger impact than stating it can open several chrome processes is a totally different thing.

j-mendez commented 1 year ago

In order for concurrent operations to happen a child process is required and within each child process managing the browser. In nodejs you can only fork separate processes https://nodejs.org/api/cluster.html since the nodejs runtime is required in each one. This makes it really not possible or meant for concurrency. You can do handle things parallel just not concurrent.

tamusjroyce commented 1 year ago

@j-mendez a new change to libuv may change that. Seems like node is getting multithreading.