search

Chicago / west-nile-virus-predictions

Algorithm to predict repeated positive results for West Nile Virus for mosquitoes captured in traps across Chicago.
MIT License
14 stars 1 forks source link

Bug - Updates are failing #46

Closed geneorama closed 7 years ago

geneorama commented 7 years ago

Looking into but for automated predictions.

geneorama commented 7 years ago

The error is again related to the weather updates. I was testing for incomplete data with the TMAX variable, because normally if one variable is missing that's a good indicator for the whole row. However, this time the three variables we use had different levels of missingness.

I've updated the code to check all three variables. Hopefully the hourly data (which is what I'm using to supplement the daily data) doesn't have the same problems down the road.

This has been difficult to test because the WNV updates are infrequent and by the time I debug the weather has been fixed. The missing data issue doesn't last very long.

The other thing to remember in the future is that the incomplete input data doesn't cause any problems until it gets to the model, at which point the error message is "unable to group by id". So, the error looks like a problem with factors being passed to glmer, and it always takes me a few minutes to realize that the error has nothing to do with "id" (which is the calculated trap id).

geneorama commented 7 years ago

Also, I noticed that I had TMIN in the weather summary twice. Fixing that also.

geneorama commented 7 years ago

I accidentally did the work in branch docs... merged into master