Closed afuetterer closed 4 years ago
Using default settings the user agent of the crawler is simply "news-please". Do we want to change this? The user agent will be logged by the sites we crawl. It would be polite to have a proper name, maybe with a reference to the project.
Using default settings the user agent of the crawler is simply "news-please". Do we want to change this? The user agent will be logged by the sites we crawl. It would be polite to have a proper name, maybe with a reference to the project.