search

unitedstates / congressional-record

A parser for the Congressional Record.
Other
128 stars 40 forks source link

Documentation--change usage limit description. #22

Closed bmschmidt closed 6 years ago

bmschmidt commented 8 years ago

This is just a minor documentation issue, but I was worried about exceeding the download limit, so I wrote the GPO to ask what the usage limit was: they say there isn't one, but ask for nighttime downloading.

Their policy:

I have consulted with our developers and the infrastructure team and they confirmed that GPO does not currently throttle downloads from FDsys. We do review traffic and take action against 'abusive' traffic. To avoid this situation, we recommend the following actions:

  • Use a descriptive user agent so that it is easier to identify and troubleshoot any potential issues.
  • Set your script to download your intended material (i.e. Congressional Record) between 8 p.m. - 6 a.m.
nclarkjudd commented 6 years ago

Make sure user-agent is descriptive whenever it's time to fix https://github.com/unitedstates/congressional-record/issues/24.

Maybe modify user-agent for each user according to a .conf file

nclarkjudd commented 6 years ago

Won't be doing anything with respect to metering usage/downloads. This version already includes incremental backoff etc. such that if used as written, it will be polite to GPO servers.