HTTP 404 Error downloading climate normals

[steffilazerte]

Hello WeatherCan Team,

First off, thank you for all the hard work you put into WeatherCan. It was the package that got me started in R and I am loving it!

Unfortunately I am also having an error when trying to download climate normals from a specific station:

n <- normals_dl(climate_ids = "6046767") Error: Problem with mutate() input normals. x Not Found (HTTP 404). Failed to access climate normals for this station (climate id: 6046767). i Input normals is purrr::map(.data$loc, normals_raw). Run rlang::last_error() to see where the error occurred.

I've tried to do some digging to help with debugging

Behaviour in R:

• When looking for the station with stations_search, it has "Normals = True"
• When searching "Normals_measurement" both stations climate ID's (6046770 and 6046767) appear to have normals data

On Climate Weather.gc.ca portal:

• Climate ID does not appear in Canadian Climate Normal CSV's (1981-2010, 1971-2000, 1961-1990)
• Searching by Name, Province, and Proximity does not yield any results
• Oddly, I can search a distance from Pukaskwa National Park (where the station is located), but it still generates no results
• There was an update to the climate.weather.gc.ca page on April 7th, 2021 (yesterday) which appeared to update the station inventory list

Basically, I'm trying to figure out Is the R library incorrect (should normals = FALSE)? Have I made a technical mistake (dependencies)? Is it an error related to the recent Climate Weather update?

Thanks!

Originally posted by @urbaingeo4455 in https://github.com/ropensci/weathercan/issues/88#issuecomment-815925026

[steffilazerte]

Hi @urbaingeo4455 I'm glad you're enjoying weathercan and so happy to hear that it inspired you to check out R!

I've moved your comment to a new issue as it was unrelated to the issue you commented under. For next time, ideally for this kind of thing you would open a new issue. Your problem relates to an error, but this issue reflects getting access to normals data from a different source so they're not the same thing, it helps us keep things organized if each issue reflects a specific problem/feature/etc.

Okay! On to the current problem. Unfortunately I wasn't able to recreate your problem. When I tried to download the normals, I got an error saying that there were not normals available. Which is also the case in the stations data frame (normals are all FALSE) and the normals_measurement data frame:

library(weathercan)
library(tidyverse)

n <- normals_dl(climate_ids = "6046767")
# Error: No stations had climate normals available

filter(stations, climate_id %in% c(6046770, 6046767))
#  A tibble: 6 x 14
#   prov  station_name      station_id climate_id WMO_id TC_id   lat   lon  elev tz       interval start   end normals
#   <chr> <chr>                  <dbl> <chr>       <dbl> <chr> <dbl> <dbl> <dbl> <chr>    <chr>    <dbl> <dbl> <lgl>  
# 1 ON    PUKASKWA NATL PA…       4041 6046770        NA NA     48.6 -86.3  192  Etc/GMT… day       1983  2005 FALSE  
# 2 ON    PUKASKWA NATL PA…       4041 6046770        NA NA     48.6 -86.3  192  Etc/GMT… hour        NA    NA FALSE  
# 3 ON    PUKASKWA NATL PA…       4041 6046770        NA NA     48.6 -86.3  192  Etc/GMT… month     1983  2005 FALSE  
# 4 ON    PUKASKWA (AUT)         10918 6046767     71750 WCJ    48.6 -86.3  208. Etc/GMT… day       1996  2012 FALSE  
# 5 ON    PUKASKWA (AUT)         10918 6046767     71750 WCJ    48.6 -86.3  208. Etc/GMT… hour      1994  2012 FALSE  
# 6 ON    PUKASKWA (AUT)         10918 6046767     71750 WCJ    48.6 -86.3  208. Etc/GMT… month     2005  2006 FALSE  

filter(normals_measurements, climate_id %in% c("6046770", "6046767"))
# A tibble: 0 x 4
# … with 4 variables: prov <chr>, station_name <chr>, climate_id <chr>, measurement <chr>

I think it's likely that your version of weathercan (and/or dplyr see #111) is out of date. You can use packageVersion("weathercan") to see what version you have (it should be v0.5.0+) and install.packages("weathercan") to update. Let me know how that goes for you!

However, the recent ECCC update does mean that the stations data frame may be out of date, so I'm going to spend some time today looking into updating weathercan to match.

[urbaingeo4455]

Hi Steffi,

You're exactly right! Although I did initially try updating my packages, I missed on the console that some (including dplyr) had failed.

I've since updated all the relevant packages and everything seems to be working correctly now.

Behavior before:

Behavior after:

Thank you very much for your help on the problem and with organizing the issue!