digipres / digipres.github.io

Auto-generated static web site digipres.org
https://www.digipres.org/
26 stars 19 forks source link

Consider following Wikimedia's user-agent policy and set a digipres.github.io specific User-Agent header #31

Open ross-spencer opened 1 year ago

ross-spencer commented 1 year ago

The Wikimedia user agent policy is here; https://meta.wikimedia.org/wiki/User-Agent_policy

I believe right now via: https://github.com/digipres/digipres.github.io/blob/a1fb93280d1b89203c2e258fe02eb0d61710846b/_sources/registries/wikidata/download-wikidata-formats.py#L36 they will see a generic python-requests header which I think just about passes their test. They do however recommend a format for User-agent: User-Agent: CoolBot/0.0 (https://example.org/coolbot/; coolbot@example.org) generic-library/0.0 which may be a nicer way to play in their sandbox. I'd recommend it for PRONOM and other downloads too if one is so inclined.

I've only done a cursory glance of the code-base here, maybe there is an agent set somewhere that I am missing. Apologies if I've read that wrong.

NB. Is this script code a dead file?

maybe one for .gitignore?