search

ropensci / weathercan

R package for downloading weather data from Environment and Climate Change Canada
https://docs.ropensci.org/weathercan
GNU General Public License v3.0
102 stars 29 forks source link

Issues updating stations list #126

Closed carlos-arnillas closed 3 years ago

carlos-arnillas commented 3 years ago

Hello I am trying to update the station list using the command: stations_dl()

But I got this message:

According to Environment Canada, Modified Date: 2021-08-02 23:34 UTC                                                                        
Warning: 2 parsing failures.
 row                                                                                                                                                                                                                                                  col                     expected actual         file
8607 Name   Province    Climate ID  Station ID  WMO ID  TC ID   Latitude (Decimal Degrees)  Longitude (Decimal Degrees) Latitude    Longitude   Elevation (m)   First Year  Last Year   HLY First Year  HLY Last Year   DLY First Year  DLY Last Year   MLY First Year  MLY Last Year delimiter or quote                     <raw vector>
8607 Name   Province    Climate ID  Station ID  WMO ID  TC ID   Latitude (Decimal Degrees)  Longitude (Decimal Degrees) Latitude    Longitude   Elevation (m)   First Year  Last Year   HLY First Year  HLY Last Year   DLY First Year  DLY Last Year   MLY First Year  MLY Last Year closing quote at end of file        <raw vector>

Error: Can't subset columns that don't exist.
x Column `Province` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.

The issue seems to be in line 213 of the stations.R file. It says:

httr::content(resp, type = "text/csv", encoding = "Latin1"

however, when I downloaded the data, I saw that the current format is tab-separated, so this simple fix may solve the issue.

httr::content(resp, type = "text/tab-separated-value", encoding = "Latin1"

Best,

Carlos Alberto

steffilazerte commented 3 years ago

Hi Carlos, thanks for the report! Yes, we saw that change, but haven't gotten the fixes on the main branch yet. We're hoping to have a fix that will catch tsv or csv in case ECCC changes the format (again!). Will let you know when that's been resolved, thanks!

boshek commented 3 years ago

You could also try installing the work in progress here using this one-liner:

devtools::install_github("ropensci/weathercan", ref = "recursive-fix")

If possible then you could report back if this has fixed your issue.

carlos-arnillas commented 3 years ago

Have you tried fread function on data.table?

On Mon, Aug 23, 2021 at 11:29 AM Sam Albers @.***> wrote:

You could also try installing the work in progress here https://github.com/ropensci/weathercan/pull/124 using this one-liner:

devtools::install_github("ropensci/weathercan", ref = "recursive-fix")

If possible then you could report back if this has fixed your issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ropensci/weathercan/issues/126#issuecomment-903876879, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGGQELIPENJWGIF5U44GPGTT6JSO7ANCNFSM5CKXAJDA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

steffilazerte commented 3 years ago

No I wasn't aware of that, we're using readr and httr, luckily all we had to do was change it from

httr::content(resp, type = "text/csv", encoding = "Latin1") httr::content(resp, type = "text", encoding = "Latin1")!

This has been pushed to the main branch so let us know if you get a chance to try it.

carlos-arnillas commented 3 years ago

The updated version fixed the problem! Thanks!