signaturescience / focustools

Forecasting COVID-19 in the US
https://signaturescience.github.io/focustools/
GNU General Public License v3.0
0 stars 0 forks source link

add r wrapper function to run python validation #9

Closed vpnagraj closed 3 years ago

vpnagraj commented 3 years ago

see https://github.com/signaturescience/focustools/issues/6

this function should wrap the validation .py script provided by C19FH:

https://github.com/reichlab/covid19-forecast-hub/blob/master/code/validation/test_formatting.py

we can call it with a system() command or reticulate (eventually?)

vpnagraj commented 3 years ago
vpnagraj commented 3 years ago

required python pkgs per https://github.com/reichlab/covid19-forecast-hub/blob/master/visualization/requirements.txt

pandas
requests
pymmwr
click
urllib3
selenium
webdriver-manager
pyyaml
PyGithub
git+https://github.com/reichlab/zoltpy/
https://github.com/hannanabdul55/pykwalify/archive/master.zip
vpnagraj commented 3 years ago

the functionality from validate_single_forecast_file.py is now implemented in two ways:

  1. by calling the python script directly from the root of this repo
  2. by using the new validate_forecast() R wrapper

digging in here and basically all that the single forecast vaildation script did was call a function from single py module (zoltpy). that was made it very easy to wrap in a function and call using reticulate.

there is more to validation though ... the other python scripts that C19FH maintains (and are now pulled into the misc/validation dir in this repo) do things like checking metadata files, looping over many forecast files, etc.

there are also the R scripts (no longer maintained by C19FH?) but pulled over and working (for now?) in this repo

we can revisit validation workflows later. for now, this wrapper function works to make sure the quantile csv format is not broken