Upgraded puppeteer to latest (22.10.0 as of writing) and puppeteer-cluster to latest (0.24.0 as of writing). The benefits of this I found are:
Version is supported by Puppeteer (the previously used version is now deprecated)
Fixes stability issues in Ubuntu 22.04 and newer
The downloaded version of Google Chrome on Apple Silicon Macs is now ARM-based, increasing performance and removing the need for the Rosetta translation layer
Concurrency limit for puppeteer-cluster is set to os.availableParallelism(), allowing computers with more CPU cores to take advantage of the available compute and putting less load on computers with less than 5 cores
When running tests, Puppeteer no longer waits a hard-coded amount of time (previously 2,200ms). It now waits in this order:
Wait for page load
Wait for network requests to finish
Wait for all intervals in the page to be cleared
Wait for all timeouts in the page to expire
Wait for the chart animation to finish
Updated line/realtime test to only run the interval twice (it only ran twice before this PR, it ran every 1000ms in a 2,200ms window, but now that Puppeteer is waiting for intervals to be cleared, a limit needed to be set to prevent Puppeteer from timing out)
In the new version of Puppeteer, page.waitFor() is no longer a function. The page.waitFor() in tests/e2e/spec/utils.js has been replaced with a promisified timeout.
In the new version of Puppeteer, the filepath for the install Chrome file is now node_modules/puppeteer/install.mjs instead of node_modules/puppeteer/install.js. This filepath change has been made to the CONTRIBUTING.md file.
Changed puppeteer-cluster concurrency from CONCURRENCY_PAGE to CONCURRENCY_BROWSER to fix various issues with concurrency after the Puppeteer upgrade
Benchmarks:
Machine 1 - M2 2023 15 inch MacBook Air 8GB Memory
Machine 2 - 6 core 2.6GHz i7 2019 16 inch MacBook Pro 32GB memory
Machine 3 - M1 2021 24 inch iMac 16GB memory
Before PR:
Machine 1 - 90s
Machine 2 - 100s
Machine 3 - 89s
After PR:
Machine 1 - 32s
Machine 2 - 64s
Machine 3 - 40s
Type of change
[x] Bug fix (non-breaking change which fixes an issue)
[x] This change requires a documentation update
Checklist:
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] New and existing unit tests pass locally with my changes
Improved end-to-end test performance by up to 3x
Changes:
Concurrency limit for puppeteer-cluster is set to
os.availableParallelism(), allowing computers with more CPU cores to take advantage of the available compute and putting less load on computers with less than 5 coresWhen running tests, Puppeteer no longer waits a hard-coded amount of time (previously 2,200ms). It now waits in this order:
Updated
line/realtimetest to only run the interval twice (it only ran twice before this PR, it ran every 1000ms in a 2,200ms window, but now that Puppeteer is waiting for intervals to be cleared, a limit needed to be set to prevent Puppeteer from timing out)In the new version of Puppeteer,
page.waitFor()is no longer a function. Thepage.waitFor()intests/e2e/spec/utils.jshas been replaced with a promisified timeout.In the new version of Puppeteer, the filepath for the install Chrome file is now
node_modules/puppeteer/install.mjsinstead ofnode_modules/puppeteer/install.js. This filepath change has been made to theCONTRIBUTING.mdfile.Changed puppeteer-cluster concurrency from
CONCURRENCY_PAGEtoCONCURRENCY_BROWSERto fix various issues with concurrency after the Puppeteer upgradeBenchmarks:
Machine 1 - M2 2023 15 inch MacBook Air 8GB Memory Machine 2 - 6 core 2.6GHz i7 2019 16 inch MacBook Pro 32GB memory Machine 3 - M1 2021 24 inch iMac 16GB memory Before PR:
After PR:
Type of change
Checklist: