search

lewisdonovan / google-news-scraper

Lightweight scraper for Google News
Other
242 stars 62 forks source link

Add support for `baseUrl` config item #44

Closed hahagu closed 8 months ago

hahagu commented 8 months ago

This has been added so that urls such as https://news.google.com/topics/CAAql... can be crawled.

Also optimizes/simplifies some url generation items.

lewisdonovan commented 8 months ago

Merged. Thanks for contributing 🙏

lewisdonovan commented 8 months ago

The different layout on Topics pages created an issue where not all articles were being scraped (when using topics). This has been fixed in the latest version, please update from NPM.

Full summary of changes in the latest changelog.