search

open-austin / azure-indigent-defense

4 stars 2 forks source link

Hash scraper #11

Closed normaljosh closed 1 year ago

normaljosh commented 1 year ago

Major changes:

  1. Stop writing files locally. Contents of the page are hashed, then written directly to the container (unless the hash/county/caseid/date combo already exists)
  2. Hash relevant part of contents on read, use that as part of filename
  3. Use logging across multiple modules as in https://docs.python.org/3/howto/logging.html#logging-from-multiple-modules rather than passing the logger as a parameter (I think this should work)

Closes #5