search

simonw / big-local-datasette

Publishing a Datasette of open projects from biglocalnews.org
https://biglocal.datasettes.com/
2 stars 0 forks source link

Keep track of ETags so I know which tables to refresh #4

Closed simonw closed 4 years ago

simonw commented 4 years ago

Track ETags used for each table in another table.

simonw commented 4 years ago

I run the HEAD requests in fetch_projects at the moment:

https://github.com/simonw/big-local-datasette/blob/c08cf9c49805c6633b8f62f66b0c443fec7505bb/fetch_projects.py#L59-L66

I should run them in populate_tables instead - that way I can use the etag column to decide if the table should be re-populated.