Closed augusto-herrmann closed 2 years ago
Based on some best practices and examples I've find, I propose
PublicBodiesBot (https://github.com/okfn/publicbodies)
Considerations:
Mozilla
bit should probably be kept out of it, considering we're not emulating a browser
Well behaved scripts and crawlers should always send a custom, non-default user agent. We should: