cmu-delphi / covidcast-indicators

Back end for producing indicators and loading them into the COVIDcast API.
https://cmu-delphi.github.io/delphi-epidata/api/covidcast.html
MIT License
12 stars 17 forks source link

Set up validation for all indicators #725

Open sgsmob opened 3 years ago

sgsmob commented 3 years ago

Indicators:

Inactive Indicators:

This should basically involve:

  1. Set up bare-bones params.json that specifies:
    • global.data_source (the indicator name)
    • global.span_length = 14
    • dynamic.smoothed_signals
    • dynamic.max_expected_lag
    • dynamic.min_expected_lag
  2. Get the indicator output by running it or downloading some cached files from somewhere
  3. Run the validation from the indicator directory using python -m delphi_validation
  4. Investigate the output and use it to tune the remaining validation parameters
qx-teo commented 3 years ago

Validation parameters have been added (and implemented) for all but one indicator ran using automation.

Exceptions include:

nmdefries commented 1 month ago

Validation is rolled out to all daily signals in dry-run mode. Acquisition blocking is turned off because validation is too sensitive for it to be useful.