datasets / publicbodies

A database of public bodies such as government departments, ministries etc.
http://publicbodies.org
MIT License
63 stars 26 forks source link

Define an user agent for scripts and crawlers #106

Closed augusto-herrmann closed 2 years ago

augusto-herrmann commented 3 years ago

Well behaved scripts and crawlers should always send a custom, non-default user agent. We should:

augusto-herrmann commented 3 years ago

Based on some best practices and examples I've find, I propose

PublicBodiesBot (https://github.com/okfn/publicbodies)

Considerations: