search

reichlab / forecast-repository

Codebase for Zoltar forecast repository
https://zoltardata.com/
GNU General Public License v3.0
6 stars 3 forks source link

Continue truth-as-forecasts refactoring #286

Closed matthewcornell closed 3 years ago

matthewcornell commented 3 years ago

Now that [Think about representing truth as an oracle model #280] is closed, with truth being represented as a behind-the-scenes oracle model with forecasts, we can continue the refactoring. Possibilities:

  1. Enable truth versioning. Currently, uploading truth first deletes existing oracle forecasts. Details:

    • uploading truth should not delete previous truth
    • however, it must check for conflicting issue_date, say when someone tries to update multiple times in one day web UI needs to take into account version. for example, it might be nice to see a list of updates/versions
    • querying truth: add as_of option. related: [Change as_of forecast query semantics to handle partial forecast uploads #284]
    • what about the issue of gigantic truth files blowing up the point prediction table? In my mind, we haven't resolved the idea of pushing incremental updates

  2. Enable multiple oracle models. This would require a user interface for managing them, plus API parameters to specify which oracle model to upload truth to.

matthewcornell commented 3 years ago

Closing and splitting into: