search

JustZack / audit-congress

A website collecting & displaying information about the activities of congress.
0 stars 0 forks source link

Revise caches to only commit AFTER all work is done #63

Closed JustZack closed 6 months ago

JustZack commented 6 months ago

right now commits occur after every operation: Drop & Insert

Since the cache scripts delete rows then inserts the new ones - in some situations tables can be left empty.

This is mainly an issue for the Bills tables, where during cache updates the most recent information is deleted before being inserted in bulk.

Need to implement means to drop then insert new data without having data be unavailable for too long.

This would also mean tables using cache scripts should wait up to n seconds for completion. If it still isn't complete by then:

  1. Fetch old data
  2. Ensure the frontend is aware it is "old" data & new info is being procured
JustZack commented 6 months ago

Only try if #64 proves to be fruitless