ropensci / GSODR

API Client for Global Surface Summary of the Day ('GSOD') Weather Data Client in R
https://docs.ropensci.org/GSODR
Other
86 stars 15 forks source link

Error in open.connection(con, "rb") : Timeout was reached #10

Closed pat-s closed 8 years ago

pat-s commented 8 years ago

Just tried your function using the following setting:

GSODR::get_GSOD(years = 1973:2015, path = "~/Downloads/", station = "080250-99999")

Failing with the error above. If I set years to 2003:2015, everything works.

Thanks for checking!

adamhsparks commented 8 years ago

Thanks, I'll check into it and see what can be done.

adamhsparks commented 8 years ago

@PRoyal22, did this repeatedly happen? I just tried the station/years you specified. It failed the first time, but the second time and third it worked fine.

It seems to be an issue with the connection, rather than the R package itself. I'll see if I can make the package fail and/or restart gracefully in these situations, but for now I'd suggest just retrying. See what year was the last downloaded and start from there.

pat-s commented 8 years ago

Thanks for checking!

Tried it three times and every time it stopped after the fourth or fifth year. Starting every time by hand manually again is quite uncomfortable but maybe sufficient.

What I do not get with the connection problem is that it repeatedly failed for a specific year range and for other years it worked (for the same station). The problem is also not due to the large number of years as the 2003:2015 example also works.

Well, nasty bug :/

adamhsparks commented 8 years ago

Thanks for the feedback. I'll keep working on it.

adamhsparks commented 8 years ago

@PRoyal22, I've made a few changes to the function that should help fix the problem.

Since I'm unable to reliably recreate this issue, could I ask you to install the version from my GitHub master branch rather than CRAN and see if I've fixed the issue?

pat-s commented 8 years ago

@adamhsparks installed via Github and it works now, thanks! Maybe a short comment on your changes?

For future development, a convenience option merging all downloaded yearly files into one .csv file would be great!

Maybe I find some time to contribute!

Regards, Patrick

adamhsparks commented 8 years ago

That's great! Thanks for the help. I do agree about merging the output files into one file. I'll open another issue for that.

I'll close this now.

adamhsparks commented 8 years ago

Fixed with commit https://github.com/adamhsparks/GSODR/commit/a126641e00dc7acc21844ff0436e5702f8b6e04a

ckumarMT commented 7 years ago

ext_tracks_file <- paste0("http://rammb.cira.colostate.edu/research/",

  • "tropical_cyclones/tc_extended_best_track_dataset/",
  • "data/ebtrk_atlc_1988_2015.txt") head(ext_tracks_file) [1] "http://rammb.cira.colostate.edu/research/tropical_cyclones/tc_extended_best_track_dataset/data/ebtrk_atlc_1988_2015.txt" library(readr) Warning message: package ‘readr’ was built under R version 3.2.5

    Create a vector of the width of each column

    ext_tracks_widths <- c(7, 10, 2, 2, 3, 5, 5, 6, 4, 5, 4, 4, 5, 3, 4, 3, 3, 3,

  • 4, 3, 3, 3, 4, 3, 3, 3, 2, 6, 1) ext_tracks_colnames <- c("storm_id", "storm_name", "month", "day",
  • "hour", "year", "latitude", "longitude",
  • "max_wind", "min_pressure", "rad_max_wind",
  • "eye_diameter", "pressure_1", "pressure_2",
  • paste("radius34", c("ne", "se", "sw", "nw"), sep = ""),
  • paste("radius50", c("ne", "se", "sw", "nw"), sep = ""),
  • paste("radius64", c("ne", "se", "sw", "nw"), sep = ""),
  • "storm_type", "distance_to_land", "final")

    Read the file in from its url

    ext_tracks <- read_fwf(ext_tracks_file,

  • fwf_widths(ext_tracks_widths, ext_tracks_colnames),
  • na = "-99") Error in open.connection(con, "rb") : Timeout was reached ext_tracks[1:3, 1:9] Error: object 'ext_tracks' not found

    Read the file in from its url

    ext_tracks <- read_fwf(ext_tracks_file, fwf_widths(ext_tracks_widths, ext_tracks_colnames),na = "-99") Error in open.connection(con, "rb") : Timeout was reached

I am trying to open one Web Based Flat file and i get Time Out error, Can any one help please? Thanks in Advance!

adamhsparks commented 7 years ago

@ckumarMT, I'm not sure what you're doing here but it doesn't look related to this issue or to the GSODR package. Are you instead using rnoaa?

GSODR does not have access to any data related to cyclones or web based flat files.

ckumarMT commented 7 years ago

I am importing the CSV file from below location using read_fwf() Function. I have created two vectors to store ColumnName and ColumnLength File Location:- http://rammb.cira.colostate.edu/research/tropical_cyclones/tc_extended_best_track_dataset/data/ebtrk_atlc_1988_2015.txt

adamhsparks commented 7 years ago

@ckumarMT, again this doesn't appear to be related to this issue or the GSODR package at all. I cannot help you as it is not applicable here.

ckumarMT commented 7 years ago

Ok Thank you it is related to readr library

amine67 commented 7 years ago

Hi I having the same problem (Error in open.connection(x, "rb") : Timeout was reached)

library(rvest) library(xml2) uci_html <- read_html("http://archive.ics.uci.edu/ml/datasets.html")

Error in open.connection(x, "rb") : Timeout was reached

adamhsparks commented 7 years ago

Hi, @amine67. Please see the immediate prior comment to yours. This error has nothing to do with the GSODR package, it's related to readr.

amine67 commented 7 years ago

Thank you ... how to overcome it ?

adamhsparks commented 7 years ago

I don't know. You're asking in the wrong place as your question is not related to the GSODR package.