testing forecasters on simple datasets

cmu-delphi / exploration-tooling

tools for evaluating and exploring forecasters

Other

0 stars 0 forks source link

testing forecasters on simple datasets #32

Closed dsweber2 closed 1 year ago

dsweber2 commented 1 year ago

addresses #23 and #20. I ended up not actually using targets for the simple data tests b/c it ended up too awkward and distributed. I should probably do a little cleanup of the repo and check for edge cases before we actually merge this.

dsweber2 commented 1 year ago

I also added the flatline forecaster and made the necessary changes to get that working. Should be ready for review (pending the tests passing on the remote).

dsweber2 commented 1 year ago

To be clear, the tests this adds are:

2 states with different constants (technically normally distributed with sd=1e-5, to avoid problems with linear_reg)
single state with white noise (sd=2 while the mean is 25)
2 states, one of which has latency varying between 0-3 days (poisson mean 0.5)
linearly increasing by 1 each day (the flatline is exempt from passing this test)

dsweber2 commented 1 year ago

output date correctness is checked here in the "basics" section, rather than the "data" section: https://github.com/cmu-delphi/exploration-tooling/blob/b60b500c11c1cce820611cdc9c275adc7ce6d47d/tests/testthat/test-forecasters-basics.R#L18-L21 as well as somewhat indirectly in the latency adjusting specific functions: https://github.com/cmu-delphi/exploration-tooling/blob/98c13702f4e81c66d48972b7f9c4793eab21e06d/tests/testthat/test-latency_adjusting.R#L3