search

dscgt / recycling_website

Admin website for GT OSWM&R.
0 stars 0 forks source link

Record retrieval time range limits #56

Open michaelyfan opened 3 years ago

michaelyfan commented 3 years ago

47, #39, and #44 have been merged, which implement time range selection for record retrieval, and these default to economic time ranges. However, users still have the option to retrieve a bunch of records from a very large time range. Retrieval of one record equates to one document read.

This request could still eventually explode. A future worst-case scenario is that over 50,000 (the Firebase free reads-per-day limit) records exist, and the user decides to download all of them, and/or does multiple downloads a day. It might take years to get to this point, but we should still prepare for this scale. Is moving beyond free tier inevitable?

Limiting reads: https://firebase.google.com/docs/firestore/query-data/order-limit-data

michaelyfan commented 3 years ago

Final solution probably combines some of these

Option 1: limit records downloading on client*. This may involve displaying something like, "Sorry, daily limit reached!". If they need a bigger download, they can request it from us. There should be a message displaying this process and its justification.

Option 2: after amount of data stored becomes too large, download all of it as Excel files, and ask recycling office to hold onto this as well. Then, wipe the database records (MODELS AND GROUPS STAY).

Option 3: don't do anything and hope that yearly credits can pay for everything

Option 4: request that the recycling office pay for it.

*limiting mechanisms: per timerange (ex. day, hour), per download (may be less effective)