blab / ncov-forecasting-fit

Assessing accuracy of fitness model forecasts
1 stars 2 forks source link

Figure 4B correlation #26

Closed trvrb closed 1 year ago

trvrb commented 1 year ago

In commit f0207d393a30125b24646f92cb7e441516ff39e5 the R correlation coefficient for proportion bad QC in Figure 4B went from 0.22 to 0.01. What's going on here?

This is good reason why informative commit messages are helpful. I can't interpret "updating figure 4". This is also why updating figure specific .ipynb along with updating PNGs is helpful.

Eabousam commented 1 year ago

Fixed issue and reran all data and figures for var of interest

Eabousam commented 1 year ago

Code used to generate the plots are in sections:

trvrb commented 1 year ago

Thanks for looking into this. What's the difference in methodology between the 0.01 result and the 0.22 result? From inspecting b9fec4770c32b1ce9c4e1dfd2d0e224d5d619a65 it looks like the only impactful change would have been swapping mae_seq_full = forecast_mae for mae_seq_full = nowcast_mae. Is this the difference between nowcast vs +30 day forecast error?

Eabousam commented 1 year ago

Yes, 0.01 result comes from filtering the lead == 30 instead of lead == 0.