Our current scheme for creating score csv files for downloading score data is showing its limitation with respect to scalability by using too much server memory. Nick told me about https://github.com/reichlab/covid19-forecast-hub/issues/1091 , which will definitely break the current csv file-based approach. He has proposed moving from files to score querying, similar to how forecast querying works, as documented at https://docs.zoltardata.com/queryformat/ , an idea I really like. I propose an API that's passed a query list/dict (R/Python) similar to forecasts:
Here I show IDs for the first four items, but we'd provide a utility to convert from strings to them, as we've done with forecast queries. Notes:
output format: data.frame same as current
we would have a row result limit like forecast data queries
for non-API users we will need some kind of ability to download scores from the web UI (no more Download button for the entire project). Solutions include a form where users can select model, unit, target, timezero, and score. A quicker-to-implement one would be a single JSON text form they could submit. Users would type in a (non-ID) query, click Run, and be taken to a standard Job detail page. They could reload the page until the job's done and then click the (new) Download Job File button.
Our current scheme for creating score csv files for downloading score data is showing its limitation with respect to scalability by using too much server memory. Nick told me about https://github.com/reichlab/covid19-forecast-hub/issues/1091 , which will definitely break the current csv file-based approach. He has proposed moving from files to score querying, similar to how forecast querying works, as documented at https://docs.zoltardata.com/queryformat/ , an idea I really like. I propose an API that's passed a query list/dict (R/Python) similar to forecasts:
Here I show IDs for the first four items, but we'd provide a utility to convert from strings to them, as we've done with forecast queries. Notes: