search

lewisdonovan / google-news-scraper

Lightweight scraper for Google News
Other
242 stars 62 forks source link

Allow setting puppeteer headless mode in userConfig #45

Closed ole-ve closed 8 months ago

ole-ve commented 8 months ago

Using headless: true seems to have a significant performance impact over using headless: 'shell'. There is an issue in the puppeteer repository and during some testing, I found using shell was around 7-8 times faster for my queries.

It would be great if one could set the puppeteer headless mode by defining it as an input argument.

ole-ve commented 8 months ago

If you find time, would be great if you could take a look @lewisdonovan (I cannot assign reviewers myself)

lewisdonovan commented 8 months ago

Great find! Tested locally and it gives a performance boost of roughly 30%. I'll merge this now and update the NPM module shortly.

lewisdonovan commented 8 months ago

Latest version now available in NPM. To update please run npm update google-news-scraper.

ole-ve commented 8 months ago

Thanks a lot for the fast merging 👍