search

Chicago / west-nile-virus-predictions

Algorithm to predict repeated positive results for West Nile Virus for mosquitoes captured in traps across Chicago.
MIT License
14 stars 1 forks source link

Bug when retrieving data in 2017 #18

Closed geneorama closed 7 years ago

geneorama commented 7 years ago

There is a download when refreshing the NOAA data, today. This will probably be temporary, and will resolve as soon as there is enough history for the year.

Currently (Jan 4 2017) the NOAA API returns a zero length list for 2017-01-01 to 2017-01-04, apparently the data isn't loaded yet, which causes an error in get_noaa_content, which is called from refresh_noaa

Moving back the date in the request one day works, last year works, so I think this will resolve itself and they simply haven't started the new year of data yet.

Nonetheless, I should put in a check for zero length content.

result <- httr::GET(paste0("http://www.ncdc.noaa.gov/cdo-web/api/v2/data?datasetid=GHCND&",
                           "stationid=GHCND:USW00094846&startdate=2017-01-01&enddate=2017-01-04&limit=1000&offset=0"),
                    httr::add_headers(token=token))
httr::content(result)

result <- httr::GET(paste0("http://www.ncdc.noaa.gov/cdo-web/api/v2/data?datasetid=GHCND&",
                           "stationid=GHCND:USW00094846&startdate=2016-12-31&enddate=2017-01-04&limit=1000&offset=0"),
                    httr::add_headers(token=token))
httr::content(result)[[2]]

image

geneorama commented 7 years ago

Done also refactored the code a bit to isolate the date portion of the request. Now refresh_noaa strictly automates the NOAA file downloading, and get_noaa_content handles the web request / URL creation. (Previously the URL was partly created in the first function). This is how it should have been in the first place, but it also made it easier to add error handling.