Timeout?

[jhollist]

Trying to grab the latest with lagosne_get("1.087.1") and the request times out.

File downloads but then returns:

Error in curl::curl_fetch_memory(url, handle) : 
  Timeout was reached: Connection timed out after 10860 milliseconds

Throws that upon line 29 in get.R.

the curl_fetch_memory in get_file_names chokes. I haven't used that before so just grasping at straws, but perhaps httr::GET and httr::write_disk might be a good replacement so as to allow passing of configs such as timeout?

That is as far as I got on a fix, because I, amusingly, ran out of time! Placing this issue to record the problem and if I can, will try to work on a solution.

[jsta]

This error pops up in a number of other packages (see https://github.com/ropensci/neotoma/issues/209 and https://github.com/jeroen/curl/issues/72). I have seen it happen on a machine behind a strict firewall. I tried to fix by increasing the timeout but had no luck.

[jhollist]

Hmm...

I am sure we don't have a permissive firewall! But it is odd that it works for the locus, but not for limno or geo. Suggests something else to me..

I'll see if I can carve out some time to look at it. If I can get it working on the EPA network it should work anywhere!

[limnoliver]

I get a lot of curl errors at the USGS. Does running the following work for you?

httr::set_config(config(ssl_verifypeer = 0L))

[jhollist]

Thanks for that suggestion, @limnoliver. Unfortunately, didn't do the trick for me.

I might be able to carve out some time to dig into this to see if I can figure out a solution. Stay tuned.

[jhollist]

Let me preface all of this with the fact that I know very little about the various data portals available for this. With that...

I did find this data also on https://portal.edirepository.org/nis/mapbrowse?packageid=edi.101.2. And that link works and you can use with the entityid to grab the filename. For instance:

headers <- curl::parse_headers(
+     curl::curl_fetch_memory("https://portal.edirepository.org/nis/dataviewer?packageid=edi.101.2&entityid=dea6da629bab1338a3d564ce3bd49fea", handle)$headers)

Is there a problem using the edirepository as opposed to (or as an option to) pasta.lternet.edu?

[jhollist]

And, I think it is working from the edi repository in https://github.com/jhollist/LAGOSNE

I'll look a bit more after the holiday.

Happy Thanksgiving!

[jsta]

Looks great! I think that should work for everyone regardless of prior lack of timeout issues. You up for submitting a PR?

[jhollist]

I am! Wanted to make sure you all were OK prior to doing that. Should be able to get to it sometime today.

[jhollist]

Just submitted a PR (#61) Now to start playing with the data (we have a paper idea we are working on and hope to use LAGOSNE for it).