openaddresses / openaddresses-ops

Issues-only repo for discussion of operational considerations for OA
6 stars 5 forks source link

Inconsistent download caches #30

Open markwoodbury opened 3 years ago

markwoodbury commented 3 years ago

In follow up to the scripts I've put in place for #29, I'm seeing inconsistent versions being returned for downloads of the collection files. As an example, see below outputs from my logs, which paste out the last-modified value of curl to https://v2.openaddresses.io/batch-prod/collection-global.zip

Sep 28 09:32:01 root: OA - Most recent = Sun, 20 Sep 2020 15:20:53 GMT Sep 28 10:32:02 root: OA - Most recent = Sun, 20 Sep 2020 15:20:53 GMT Sep 28 11:32:01 root: OA - Most recent = Sun, 27 Sep 2020 15:16:54 GMT Sep 28 20:32:01 root: OA - Most recent = Sun, 20 Sep 2020 15:20:53 GMT Sep 28 23:32:01 root: OA - Most recent = Sun, 27 Sep 2020 15:16:54 GMT Sep 29 05:32:02 root: OA - Most recent = Sun, 20 Sep 2020 15:20:53 GMT Sep 29 08:32:01 root: OA - Most recent = Sun, 27 Sep 2020 15:16:54 GMT

Can you please confirm if the download caches are purged when a new file is published? Probably not too much of an issue for me now I know this may happen, but could result in out of date files being served.