Undefined error in wdpa_fetch()

[stanleesocca]

Hi,

Thanks for the awesome package. I was trying to use the package to download MPA data from France but on running wdpa_fetch() I was greeted with an error. Can you help with that.

library(sf)
library(wdpar)

wdpaid <- "555526224"
mpa <- as_Spatial(wdpa_fetch("France")) %>% filter(WDPAID == wdpaid)

Error shown: Error in checkError(res) : Undefined error in httr call. httr output: Failed to connect to localhost port 4567: Connection refused

Can you explain how I can solve this issue.

[jeffreyhanson]

Hey, I'm sorry I don't know of any immediate fixes off the top of my head. The code for downloading the data uses a virtual web browser to download the code and sometimes it can fail to correctly shut down the web browser when it finishes, which causes problems when you try and download more data. Could you please try restarting your computer and trying the code again (this will terminate the virtual web browser if it's still running)? Also, could you please type in traceback() after running the code and copy in the output messages here so I can exactly exactly where it's going wrong?

[stanleesocca]

Thanks for the reply. I tried the first option of restarting my computer but it didn't work. The traceback() result:

15: stop(err)
14: checkError(res)
13: queryRD(qpath, "POST", qdata = serverOpts)
12: rd$open(silent = TRUE)
11: tryCatchList(expr, classes, parentenv, handlers)
10: tryCatch({
        url <- character(0)
        pjs <- wdman::phantomjs(verbose = FALSE)
        rd <- RSelenium::remoteDriver(port = 4567L, browserName = "phantomjs")
        rd$open(silent = TRUE)
        rd$maxWindowSize()
        rd$navigate(paste0("https://protectedplanet.net/country/", 
            x))
        Sys.sleep(2)
        elem <- rd$findElement(using = "css", ".link-with-icon--bold")
        elem$clickElement()
        Sys.sleep(2)
        elem <- rd$findElement(using = "css", ".link-with-icon+ .link-with-icon")
        elem$clickElement()
        Sys.sleep(2)
        src <- xml2::read_html(rd$getPageSource()[[1]][[1]], encoding = "UTF-8")
        divs <- xml2::xml_find_all(src, ".//div")
        divs <- divs[which(xml2::xml_attr(divs, "id") == "download-modal")]
        attrs <- xml2::xml_attr(xml2::xml_find_all(divs, ".//a"), 
            "href")
        url <- grep("shapefile", attrs, fixed = TRUE, value = TRUE)
    }, finally = {
        try(rd$close(), silent = TRUE)
        try(rd$close(), silent = TRUE)
        try(pjs$stop(), silent = TRUE)
        try(pjs$stop(), silent = TRUE)
    })
9: withCallingHandlers(expr, message = function(c) invokeRestart("muffleMessage"))
8: suppressMessages(tryCatch({
       url <- character(0)
       pjs <- wdman::phantomjs(verbose = FALSE)
       rd <- RSelenium::remoteDriver(port = 4567L, browserName = "phantomjs")
       rd$open(silent = TRUE)
       rd$maxWindowSize()
       rd$navigate(paste0("https://protectedplanet.net/country/", 
           x))
       Sys.sleep(2)
       elem <- rd$findElement(using = "css", ".link-with-icon--bold")
       elem$clickElement()
       Sys.sleep(2)
       elem <- rd$findElement(using = "css", ".link-with-icon+ .link-with-icon")
       elem$clickElement()
       Sys.sleep(2)
       src <- xml2::read_html(rd$getPageSource()[[1]][[1]], encoding = "UTF-8")
       divs <- xml2::xml_find_all(src, ".//div")
       divs <- divs[which(xml2::xml_attr(divs, "id") == "download-modal")]
       attrs <- xml2::xml_attr(xml2::xml_find_all(divs, ".//a"), 
           "href")
       url <- grep("shapefile", attrs, fixed = TRUE, value = TRUE)
   }, finally = {
       try(rd$close(), silent = TRUE)
       try(rd$close(), silent = TRUE)
       try(pjs$stop(), silent = TRUE)
       try(pjs$stop(), silent = TRUE)
   }))
7: try_and_find_url(x)
6: wdpa_url(x, wait = wait)
5: wdpa_fetch("France")
4: as_Spatial(wdpa_fetch("France"))
3: eval(lhs, parent, parent)
2: eval(lhs, parent, parent)
1: as_Spatial(wdpa_fetch("France")) %>% filter(WDPAID == wdpaid)

Hope this will help with the bug.

[jeffreyhanson]

Thanks! Yeah, it looks like something strange is happening with the virtual web browser (Phantom JS). This could be due firewall stuff - I don't know sorry. In terms of getting a working solution on your system, you could try something like the code below where we hard-code the download URL:

# load pacakges
library(sf)
library(wdpar)

# set parameters
wdpaid <- "555526224"
download_url <-  "https://www.protectedplanet.net/downloads/WDPA_Aug2019_FRA?type=shapefile"
download_dir <- rappdirs::user_data_dir("wdpar")

# download the file
file_name <- basename(httr::HEAD(download_url)$url)
file_path <- file.path(download_dir, file_name)
curl::curl_download(download_url, file_path, quiet = FALSE)

# read the data
france_pa <- wdpa_read(file_path)

# subset the data
mpa <- france_pa %>% filter(WDPAID == wdpaid)

Please let me know if you have any problems with that. I'm working on a mobile phone hotspot connection at the moment so I can't test it myself.

[stanleesocca]

Cool. It works fine. Thanks for the help. Hopefully you get to fit the virtual browser issue and wdpa_fetch() pipeline as I imagine with such way means less code.

Thanks once again. Hopefully, one day I will understand all these stuff well enough to solve some of them.

[jeffreyhanson]

No worries - glad I could help.