search

ThreeSixtyGiving / datagetter

Scripts to download data from http://registry.threesixtygiving.org
MIT License
1 stars 1 forks source link

get: Fix the way in which we guess the file_type #27

Closed michaelwood closed 3 years ago

michaelwood commented 3 years ago

If we get strange values from the http header that comes back from fetching a file with an incomplete content-disposition entry we end up with None as the value for the filename which then breaks the file_type guessing.

  1. Make sure we can handle this issue
  2. Fall back to using the file extension in the url
  3. Make sure the fetch_and_convert function doesn't bring down the datagetter process

Co-authored-by: Bibiana Cristofol Amat bcristofol@gmail.com

michaelwood commented 3 years ago

Additional handling of how we determine file_types after one publisher's file failed on 10/11/21