search

rmendels / rerddapXtracto

xtractomactic using rerddap
Other
14 stars 4 forks source link

Curl time out errors #9

Closed sloredo214 closed 5 years ago

sloredo214 commented 6 years ago

I have an animal tracks dataset of 11876 rows and keep getting a curl time out error while extracting a wind vector dataset ("ncdcOw6hrP_LonPM180") . I am hardwired into the internet so I don't believe it is my server, so I am wondering if perhaps my dataset is too large and I should consider splitting my dataset by bird or year? I tried a subset of my data , ~200 rows, to check if my input formatting was correct and it is but I would rather not subset the data by such small chunks. Could there be something else going on besides a server issue when getting a curl time out error?

I have also been using the xtractomatic package and have not had the same issue. However, I've been having it a lot with the rerddapXtracto package.

Thanks.

rmendels commented 6 years ago

If you were running in the last day or two, our servers were down on several occasions in order to install new versions of ERDDAP. I would say try again. Not subsets of 200, maybe 3 subsets Also, the newest version, if you have it (and I assume you do), should return what it has gotten so far. And the speed of your line isn't the only determining factor of whether there are timeouts.

We also found out today that the ERDDAP update broke the ability of rerddap to deal with datasets where the latitudes run north to south. rerddapXtracto depends on rerddap, so. I reported this to the author of rerddap, and where in his code it breaks. So for now avoid any dataset with that property

Anyway try again, and let me know. If it keeps failing on small extracts, send me the code and a .RData file with the necessary time/lat/lon and I will look to see what is going on.

And do read the caveats in the README on the Github page.

-Roy

On Jan 25, 2018, at 7:51 PM, sloredo214 notifications@github.com wrote:

I have an animal tracks dataset of 11876 rows and keep getting a curl time out error while extracting a wind vector dataset ("ncdcOw6hrP_LonPM180") . I am hardwired into the internet so I don't believe it is my server, so I am wondering if perhaps my dataset is too large and I should consider splitting my dataset by bird or year? I tried a subset of my data , ~200 rows, to check if my input formatting was correct and it is but I would rather not subset the data by such small chunks. Could there be something else going on besides a server issue when getting a curl time out error?

I have also been using the xtractomatic package and have not had the same issue. However, I've been having it a lot with the rerddapXtracto package.

Thanks.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

"The contents of this message do not reflect any position of the U.S. Government or NOAA."

Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS Environmental Research Division Southwest Fisheries Science Center Note new street address 110 McAllister Way Santa Cruz, CA 95060 Phone: (831)-420-3666 Fax: (831) 420-3980 e-mail: Roy.Mendelssohn@noaa.gov www: http://www.pfeg.noaa.gov/

"Old age and treachery will overcome youth and skill." "From those who have been given much, much will be expected" "the arc of the moral universe is long, but it bends toward justice" -MLK Jr.