ipeaGIT / aopdata

Download data from the Access to Opportunities Project (AOP)
https://ipeagit.github.io/aopdata/
13 stars 4 forks source link

alert for internet connection problem #26

Closed rafapereirabr closed 3 years ago

rafapereirabr commented 3 years ago

I just got this from the CRAN team:

Dear maintainer,

Please see the problems shown on https://cran.r-project.org/web/checks/check_results_aopdata.html.

Please correct before 2021-04-12 to safely retain your package on CRAN.

It seems we need to remind you of the CRAN policy:

'Packages which use Internet resources should fail gracefully with an informative message if the resource is not available or has changed (and not give a check warning nor error).'

This needs correction whether or not the resource recovers.

The CRAN Team

and a secont message from them:

I have seen several different failures in the last few days:

--- re-building ‘access_maps.Rmd’ using rmarkdown Linking to GEOS 3.8.1, GDAL 3.0.4, PROJ 6.3.2 Quitting from lines 36-43 (access_maps.Rmd) Error: processing vignette 'access_maps.Rmd' failed with diagnostics: object 'type' not found --- failed re-building ‘access_maps.Rmd’

--- re-building ‘landuse_maps.Rmd’ using rmarkdown Linking to GEOS 3.8.1, GDAL 3.0.4, PROJ 6.3.2 Quitting from lines 35-42 (landuse_maps.Rmd) Error: processing vignette 'landuse_maps.Rmd' failed with diagnostics: Cannot open "/tmp/RtmprkkzTh/working_dir/RtmpKy82ey/hex_grid_for.gpkg"; The source could be corrupt or not supported. See st_drivers() for a list of supported formats. --- failed re-building ‘landuse_maps.Rmd’

--- re-building ‘access_maps.Rmd’ using rmarkdown Linking to GEOS 3.9.0, GDAL 3.2.1, PROJ 6.3.2 Using year 2019 Using mode public_transport Quitting from lines 36-43 (access_maps.Rmd) Error: processing vignette 'access_maps.Rmd' failed with diagnostics: Timeout was reached: [www.ipea.gov.br] Connection timed out after 10013 milliseconds --- failed re-building ‘access_maps.Rmd’

And that website currently also has

Check: whether package can be installed Result: WARN Found the following significant warnings: Warning: unable to re-encode 'utils.R' line 334

(checking in Latin-1) and

 --- re-building ‘landuse_maps.Rmd’ using rmarkdown
 Linking to GEOS 3.9.0, GDAL 3.2.1, PROJ 7.2.1
 Quitting from lines 35-42 (landuse_maps.Rmd)
 Error: processing vignette 'landuse_maps.Rmd' failed with diagnostics:
 some columns are not in the data.table:

abbrev_muni,name_muni,code_muni,id_hex --- failed re-building ‘landuse_maps.Rmd’

--- re-building ‘landuse_maps.Rmd’ using rmarkdown Warning in engine$weave(file, quiet = quiet, encoding = enc) : Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to R Markdown v1. Linking to GEOS 3.6.4, GDAL 2.2.4, PROJ 5.2.0 Quitting from lines 35-42 (landuse_maps.Rmd) Error: processing vignette 'landuse_maps.Rmd' failed with diagnostics: Timeout was reached: [www.ipea.gov.br] Operation timed out after 10140 milliseconds with 0 out of 0 bytes received --- failed re-building ‘landuse_maps.Rmd’

-- Brian D. Ripley,

rafapereirabr commented 3 years ago

Ok, I've now created / organized a new function is_online() to check the Internet connection with Ipea server. See commit f8f1d4f. This will improve the error messages, but it does not entirely solve the problem, which seems to involve making changes to Ipea's server infrastructure. I'm checking this with our IT team.

dhersz commented 3 years ago

CRAN will complain about options setting. If this is necessary, we should use on.exit(options(warn = oldw)) right after setting it. But is it really necessary if we're already using suppressWarnings()?

rafapereirabr commented 3 years ago

For some reason, suppressWarnings() was not working in some cases when I first implemented in on geobr. I've fixed the options by adding on.exit(options(warn = oldw)) as you suggested. See a74f7b1.

rafapereirabr commented 3 years ago

Ok. I've added one internet connection test to download the metadata file and another one to download the proper data set. No noticeble damage to download performance. Closing this issue in my next comit

rafapereirabr commented 3 years ago

I've updated the function to make it simpler and to follow CRAN policies. CRAN requires "fail gracefully with an informative message if the resource is not available or has changed (and not give a check warning nor error)."

#' Check internet connection with Ipea server
#'
#' @description
#' Checks if there is internet connection to Ipea server to download aop data.
#'
#' @param file_url A string with the file_url address of an aop dataset
#'
#' @return Logic `TRUE or `FALSE`.
#'
#' @export
#' @family support functions
#'
check_connection <- function(file_url = 'https://www.ipea.gov.br/geobr/aopdata/metadata/metadata.csv'){

  # check internet connection
  if (!curl::has_internet()) {
    message("No internet connection.")
    return(invisible(NULL))
  }

  # test server connection
    # crul::ok(file_url)
  if (httr::http_error(httr::GET(file_url))) {
    message("Problem connecting to data server. Please try aopdata again in a few minutes.")
    return(invisible(NULL))
    }
}