blab / ncov-forecasting-fit

Assessing accuracy of fitness model forecasts
1 stars 2 forks source link

Figure 4B correlation #26

Closed trvrb closed 10 months ago

trvrb commented 10 months ago

In commit f0207d393a30125b24646f92cb7e441516ff39e5 the R correlation coefficient for proportion bad QC in Figure 4B went from 0.22 to 0.01. What's going on here?

This is good reason why informative commit messages are helpful. I can't interpret "updating figure 4". This is also why updating figure specific .ipynb along with updating PNGs is helpful.

Eabousam commented 10 months ago

Fixed issue and reran all data and figures for var of interest

Eabousam commented 10 months ago

Code used to generate the plots are in sections:

trvrb commented 10 months ago

Thanks for looking into this. What's the difference in methodology between the 0.01 result and the 0.22 result? From inspecting b9fec4770c32b1ce9c4e1dfd2d0e224d5d619a65 it looks like the only impactful change would have been swapping mae_seq_full = forecast_mae for mae_seq_full = nowcast_mae. Is this the difference between nowcast vs +30 day forecast error?

Eabousam commented 10 months ago

Yes, 0.01 result comes from filtering the lead == 30 instead of lead == 0.