search

ThreeSixtyGiving / datagetter

Scripts to download data from http://registry.threesixtygiving.org
MIT License
1 stars 1 forks source link

Store more metadata about datagetter fetch #31

Open michaelwood opened 2 years ago

michaelwood commented 2 years ago

Look at etag and last-modified http response headers to work out if a file has changed. Datastore related.

Last-Modified: ETag:

michaelwood commented 2 years ago

Insights not expiring cached files

michaelwood commented 2 years ago

Of 398 files 87 didn't provide last-modified in the header. Most of these are google docs or special feeds.