When downloading larger files, I will often timeout

[jbenzaquen42]

When downloading some larger or more complex areas, I will sometimes get the error below. Is that a limitation of the api's or is something else going on that I am missing. See example below

 .lat <- 34.086210
 .long <- 139.522432
 jima <- plot_3d_vista(lat = .lat, long = .long, phi=30
+                         , zscale = 2.5, radius = 10000)

_Downloading overlay...
Error in curl::curl_download(url = q, destfile = outfile) : 
  Timeout was reached: [] Resolving timed out after 10000 milliseconds_

[h-a-graham]

Yeah, I get this every so often - I think this is occurring somehwere within the {maptiles} package. I guess, it will happen every now and again depending on bandwidth, how busy the server is and probably other things that I am not familiar with. I just tried this and was able to download. Does this happen every time for you at this location or is it just now and again? Thanks very much for letting me know about this. Will leave this open for a while and keep an eye on things...

[jbenzaquen42]

Yeah, I get this every so often - I think this is occurring somehwere within the {maptiles} package. I guess, it will happen every now and again depending on bandwidth, how busy the server is and probably other things that I am not familiar with. I just tried this and was able to download. Does this happen every time for you at this location or is it just now and again? Thanks very much for letting me know about this. Will leave this open for a while and keep an eye on things...

Only occasionally. I was able to download r=8000 with no issues. Usually happens with files with r>10,000. I have not been able to get 15,000 to run yet at any location

[h-a-graham]

Okay that's good to know. Can you get 15000 to work with a lower detail setting?

[jbenzaquen42]

I get something like this at defullt settings at Saint Helens:

It will run up to

overlay_detail = 12, elevation_detail = 12

Before getting the timeout error

[h-a-graham]

Okay, thanks for that. I'll take a look through the maptiles code, see what might be happening, perhaps we might be able to set a longer wait time... I'll keep you posted. Cheers!

[h-a-graham]

I'm struggling to reproduce this. However, it has happened to me a few times before. Having a look through maptiles and curl::curl_download which is used to download the tiles, I don't see an obvious way to increase wait times. However, the caching system from {maptiles} is awesome so I'm wondering if we could try out something where we retry a few times and this should hopefully only download the necessary tiles. I'll try to get to this in the not too distant future!

[h-a-graham]

Okay, curiosity got the better of me. Wouldd you mind trying something for me?

I've used purrr::insistently to manage repeated attempts to download the tiles. Not being able to test this myself I can't be 100% sure it'll work but it does still return tiles as expected but I'm not getting the curl errors.

Can you install the dev branch like so:

devtools::install_github("h-a-graham/rayvista", dependencies=TRUE, ref='DEV')

And then have another go, please?

Cheers!

[jbenzaquen42]

No go. I ran this:

.lat <- 34.085641
.long <- 139.524234

helens <- plot_3d_vista(lat = .lat, long = .long, phi=30
                        , zscale = 3, radius = 15000)

And got this error a number of times before same curl error.

Error in x$.self$finalize() : attempt to apply non-function
Error in x$.self$finalize() : attempt to apply non-function
Error in x$.self$finalize() : attempt to apply non-function
Error in x$.self$finalize() : attempt to apply non-function
NULL
Error in curl::curl_fetch_disk(url, x$path, handle = handle) : 
  Timeout was reached: [s3.amazonaws.com] Resolving timed out after 10000 milliseconds
In addition: Warning message:
In showSRID(uprojargs, format = "PROJ", multiline = "NO", prefer_proj = prefer_proj) :
  PROJ/GDAL PROJ string degradation in workflow
 repeated warnings suppressed
 Discarded ellps WGS 84 in Proj4 definition: +proj=merc +a=6378137 +b=6378137 +lat_ts=0 +lon_0=0 +x_0=0 +y_0=0 +k=1 +units=m +nadgrids=@null +wktext +no_defs +type=crs

[h-a-graham]

hmm okay... So weird thing here is it looks like the overlay download actually worked and it is now failing on the DEM download. I have similarly added the insistently function to the dem download also adn bumped up the number of attemps. Oh and for your example, I'd recomend using 'fill_holes=FALSE' as there is a lot of ocean and therefore NA values so the NA fill function takes forever.

If you have time, do you mind giving it another go, please? If it fails I'll have to come back to it. Oh and it might be worth creating a local cache folder in your project - then you can check if the overlay has correctly downloaded.

The following works with the DEV version for me... fingers crossed!

devtools::install_github("h-a-graham/rayvista", dependencies=TRUE, ref='DEV')
library(rayvista)
.lat <- 34.085641
.long <- 139.524234

jima <- plot_3d_vista(lat = .lat, long = .long, phi=30
                        , zscale = 3, radius = 15000, fill_holes=FALSE, 
                        cache_dir = 'cache')

[jbenzaquen42]

Success on that one! I did mine without the cache folder. It timed out twice then went through on the 3rd retry

[h-a-graham]

Awesome! Thanks for being so quick! I'll merge this to the main branch when I get the chance before closing this issue. Look forward to seeing what you create next!

[h-a-graham]

Now merged this to main: https://github.com/h-a-graham/rayvista/pull/10

If the issue comes up again, please reopen. Thanks for the help.