Hey!
I've previously used Puppeteer for web scraping and I am looking into pre-rendering a few thousand pages in a project. I see this isn't recommended in the README.md due to RAM usage however I don't believe Puppeteer used much RAM even when scraping 50+ pages every second.
I can see that third party pages can be intercepted and blocked but is there a reason why images, media and fonts aren't intercepted and blocked? Those might be the cause of the large RAM usage and shouldn't be needed to render the page properly. Doing so would also potentially speed up the pre-rendering as the data doesn't have to be loaded.
Hey! I've previously used Puppeteer for web scraping and I am looking into pre-rendering a few thousand pages in a project. I see this isn't recommended in the README.md due to RAM usage however I don't believe Puppeteer used much RAM even when scraping 50+ pages every second.
I've tried to wrap my head around this project, but puppeteer is imported and controlled from this file in another repo, is this correct? https://github.com/JoshTheDerf/prerenderer/blob/master/renderers/renderer-puppeteer/es6/renderer.js
I can see that third party pages can be intercepted and blocked but is there a reason why images, media and fonts aren't intercepted and blocked? Those might be the cause of the large RAM usage and shouldn't be needed to render the page properly. Doing so would also potentially speed up the pre-rendering as the data doesn't have to be loaded.