The following JSON files contain data collections in array formats:
/web/data/clusters.json
/web/data/perClusterData.json
/web/data/perCountryData.json
/web/data/perCountryDataCaseCounts.json
Currently, the entire array needs to be downloaded every time, even if only some items are needed. In fact, the full download is always happening on page load, because the static import is used (see https://github.com/hodcroftlab/covariants/issues/302). This worked well in the beginning and we all hoped that pandemic is almost over back then. However, after more than a year, the data sizes have grown and mono-array is not an adequate solution anymore.
We could split the arrays into elements and allow the app to operate on individual items ("chunks"). Due to a sheer volume, we've already done this for acknowledgements data. Now it's time to transition the rest of data.
Steps:
[ ] adjust web data generation to produce separate JSON files instead of one large file with an array, plus an index file enumerating identifiers for every chunk
[ ] adjust app logic to handle chunked data, instead of iteration over an array
The following JSON files contain data collections in array formats:
/web/data/clusters.json/web/data/perClusterData.json/web/data/perCountryData.json/web/data/perCountryDataCaseCounts.jsonCurrently, the entire array needs to be downloaded every time, even if only some items are needed. In fact, the full download is always happening on page load, because the static
importis used (see https://github.com/hodcroftlab/covariants/issues/302). This worked well in the beginning and we all hoped that pandemic is almost over back then. However, after more than a year, the data sizes have grown and mono-array is not an adequate solution anymore.We could split the arrays into elements and allow the app to operate on individual items ("chunks"). Due to a sheer volume, we've already done this for acknowledgements data. Now it's time to transition the rest of data.
Steps:
This makes the most sense after or together with https://github.com/hodcroftlab/covariants/issues/302, because it will revamp the whole web app logic. We don't want to do this twice.