Problem connecting to data server is still happening

[mgkulik]

I tried for the third time in the last hour to run the function read_state with at least 15 minutes apart with the same error:

states <- geobr::read_state(year = 2020) SSL certificate problem: unable to get local issuer certificate Problem connecting to data server. Please try geobr again in a few minutes. Error in curl::curl_fetch_disk(url, x$path, handle = handle) : SSL certificate problem: unable to get local issuer certificate

I already re-installed the library but the issue persists. Waiting to execute seems also not to be the solution. Any ideas on how to solve this?

Thanks in advance, Mariane

[rafapereirabr]

Hi Mariane.

I've checked our servers and everything seems to be working fine. I suspect your network might be blocking the connection to our server. Could you please try doing this from another internet network? obs. Are you trying to do this connected to eduroam?

[mgkulik]

Thanks for the fast answer Rafael,

I'm not using eduroam and tried from two different IPs from Germany and one from US, with the same error. I'm not able to test from a Brazilian IP right now, but I even tried to use a kaggle notebook, same error.

[jhenigonsalves]

I'm getting a similar error: URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)> and I'm using a Brazilian IP, I already reinstalled the package, but it isn't working yet. This error is persisting for a few days. But, before, it worked just fine.

[rafapereirabr]

Are you using the R package or the Python package?

[lcjr86]

I'm using the python one and I still have the same issue. I don't have any VPN or firewall on my side.

[rafapereirabr]

The R package seems to be working fine. It looks like an issue on the Python side of things. @JoaoCarabetta , could you please have a look at this ?

[jhenigonsalves]

I'm using Python too, but I use a VPN, because of work, but even before, I already used it.

[mgkulik]

I'm using the R package, still not working.

[machiavelli-a]

I have a similar error. I'm using R on ubuntu and a Brazilian IP:

geobr::read_state(year = 2020) SSL certificate problem: unable to get local issuer certificate Problem connecting to data server. Please try geobr again in a few minutes. Error in curl::curl_fetch_disk(url, x$path, handle = handle) : SSL certificate problem: unable to get local issuer certificate

I've used the package before, and it worked just fine. I've had this issue in the past couple of days.

[rpietro]

Facing exactly the same issue, it was working until a few days ago.

[cmarquesluizgustavo]

Having the same problem with the r package, it was working until Dec 23. I am using a Brazillian IP.

[rpietro]

Having the same problem with the r package, it was working until Dec 23. I am using a Brazillian IP.

Exactly, I tried both with a Brazilian and US-based ip, getting the same error message on both

[rafapereirabr]

Hi all. I'm looking into this to check what could be causing the problem. In the meantime, could you please update geobr and its package dependencies to their latest version and give it a another try?

[ragapack]

Hi, First I got this error message:

SSL certificate problem: unable to get local issuer certificate
Problem connecting to data server. Please try geobr again in a few minutes.
Warning: Error in curl::curl_fetch_disk: SSL certificate problem: unable to get local issuer certificate
  57: curl::curl_fetch_disk
  56: request_fetch.write_disk
  54: request_perform
  53: httr::GET
  52: download_metadata
  51: select_metadata
  50: read_municipality
Error in curl::curl_fetch_disk(url, x$path, handle = handle) : 
  SSL certificate problem: unable to get local issuer certificate
Error in readRDS(file) : error reading from connection

Now I have an other error message:

> Brasil <- read_municipality(year=2018, code_muni="GO") 
Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  no lines available in input

[rafapereirabr]

Hi, First I got this error message:

SSL certificate problem: unable to get local issuer certificate
Problem connecting to data server. Please try geobr again in a few minutes.
Warning: Error in curl::curl_fetch_disk: SSL certificate problem: unable to get local issuer certificate
  57: curl::curl_fetch_disk
  56: request_fetch.write_disk
  54: request_perform
  53: httr::GET
  52: download_metadata
  51: select_metadata
  50: read_municipality
Error in curl::curl_fetch_disk(url, x$path, handle = handle) : 
  SSL certificate problem: unable to get local issuer certificate
Error in readRDS(file) : error reading from connection

Now I have an other error message:

> Brasil <- read_municipality(year=2018, code_muni="GO") 
Error in read.table(file = file, header = header, sep = sep, quote = quote,  : 
  no lines available in input

What did you do to get the new error message?

Given your latest error message, I believe restarting your R session will solve the problem

[jhenigonsalves]

Hi all. I'm looking into this to check what could be causing the problem. In the meantime, could you please update geobr and its package dependencies to their latest version and give it a another try?

I need to keep geopandas==0.7.0, and not the last version. The last time I tried to update it the geobr showed an error. Should I really try to update?

[ragapack]

Hi, This is the sequence after restart R

> library(geobr) 
Loading required namespace: sf 
> Estados <- read_state() 
SSL certificate problem: unable to get local issuer certificate Problem connecting to data server. 
Please try geobr again in a few minutes. Error in curl::curl_fetch_disk(url, x$path, handle = handle) :    SSL certificate problem: unable to get local issuer certificate 
> Estados <- read_state() 
Error in read.table(file = file, header = header, sep = sep, quote = quote,  :    no lines available in input

[ragapack]

It seems it's a problem on the map server.

> check_connection(file_url = "https://www.ipea.gov.br/geobr/metadata/metadata_gpkg.csv")
SSL certificate problem: unable to get local issuer certificate
Problem connecting to data server. Please try geobr again in a few minutes.

But I can download the .csv file with my browser without any problem

Direct load gpkg

> load_gpkg("https://www.ipea.gov.br/geobr/data_gpkg/state/2000/11state_2000.gpkg") 
Error in st_read.default(temps, quiet = T) :    no st_read method available for objects of class NULL

[rafapereirabr]

Are any of you using Windows? Or are you all using Linux?

[rpietro]

Linux

[ragapack]

Linux too

[luiz-dejesus]

I'm using MacOS

[mgkulik]

I tried using Ubuntu 20.10 and 20.04.

I reinstalled geobr and all its dependencies in my 20.04:

install.packages("geobr") also installing the dependencies ‘sys’, ‘utf8’, ‘bit’, ‘purrr’, ‘prettyunits’, ‘proxy’, ‘triebeard’, ‘askpass’, ‘glue’, ‘ellipsis’, ‘pkgconfig’, ‘vctrs’, ‘fansi’, ‘pillar’, ‘bit64’, ‘tidyselect’, ‘withr’, ‘progress’, ‘e1071’, ‘wk’, ‘R6’, ‘urltools’, ‘httpcode’, ‘jsonlite’, ‘mime’, ‘openssl’, ‘cli’, ‘clipr’, ‘crayon’, ‘hms’, ‘rlang’, ‘tibble’, ‘vroom’, ‘lifecycle’, ‘cpp11’, ‘tzdb’, ‘classInt’, ‘DBI’, ‘magrittr’, ‘Rcpp’, ‘s2’, ‘units’, ‘crul’, ‘curl’, ‘data.table’, ‘httr’, ‘readr’, ‘sf’

library(geobr) Loading required namespace: sf states <- geobr::read_state(year = 2020) SSL certificate problem: unable to get local issuer certificate Problem connecting to data server. Please try geobr again in a few minutes. Error in curl::curl_fetch_disk(url, x$path, handle = handle) : Timeout was reached: [www.ipea.gov.br] Connection timed out after 10000 milliseconds

[cmarquesluizgustavo]

Are any of you using Windows? Or are you all using Linux?

I am using Linux based docker containers.

[clauciorank]

Same issue Linux, if disable the ssl_verifypeer before run the read_state function the map is downloaded but the print error still happening, maybe it's a way. I'll test input this setting inside read_state or download_gpkg functions

httr::set_config(httr::config(ssl_verifypeer = 0L)) state <- geobr::read_state()

[rafapereirabr]

Coul

Same issue Linux, if disable the ssl_verifypeer before run the read_state function the map is downloaded but the print error still happening, maybe it's a way. I'll test input this setting inside read_state or download_gpkg functions

httr::set_config(httr::config(ssl_verifypeer = 0L)) state <- geobr::read_state()

@clauciorank , could please try running the code below?

metadata_link <- 'https://www.ipea.gov.br/geobr/metadata/metadata_gpkg.csv'

httr::GET(url= metadata_link, httr::config(ssl_verifypeer = 0L))

[clauciorank]

Yes it worked Response [https://www.ipea.gov.br/geobr/metadata/metadata_gpkg.csv] Date: 2021-12-29 13:53 Status: 200 Content-Type: text/csv Size: 356 kB geo,year,code,download_path,code_abrev amazonia_legal,2012,am,https://www.ipea.gov.br/geobr/data_gpkg/amazonia_legal/2012/amazonia_legal.gpkg,amazonia_legal

without the parameter httr::config(ssl_verifypeer = 0L) it returns the curl error of SSL certificate problem

[rafapereirabr]

Great. This is how we'll be fixing the R package, by telling the httr calls to ignore the SSL certificate when downloading geobr data. This poses no threat to internet security so it looks like a good solution. Thanks @clauciorank for your help !

[clauciorank]

Cool!!! It's a pleasure, this is a very useful package!!

[rafapereirabr]

For the R users on Linux

I've pushed some changes to the dev version that should fix this problem. Please install the dev version with the code below and let us know if it's working.

# install dev version
utils::remove.packages('geobr')
devtools::install_github("ipeaGIT/geobr", subdir = "r-package")
library(geobr)

# a test
s <- read_state(code_state = 'AM')

For the Python users

We already know the source of the problem, so this should be fixed soon. I'll keep you posted.

[clauciorank]

Worked fine here, it gives (warning)error message for SSL certificate, but download the map.

"Warning is not error" Plato, 400 B.C.

[ragapack]

Geat, work's for me too.

[rafapereirabr]

For the R users

Ok, so this issues seems to have been completely solved for the R users. A new version will be submitted to CRAN on Jan 3rd when CRAN will be open again for submissions. For now, please install de dev version from github.

For the Python users

We have done a few updates on our servers. Could you please let us know if the issue persists ?

[luiz-dejesus]

Hello everyone and hello @rafapereirabr,

I tested from my side (Python - MacOS) and works. I only got that as a 'warning'

ShapelyDeprecationWarning: __len__ for multi-part geometries is deprecated and will be removed in Shapely 2.0. Check the length of thegeomsproperty instead to get the number of parts of a multi-part geometry.

Thank you for the effort to fix the issue. I hope that this 'warning' will not block us in the future ;-)

Best regards,

[rafapereirabr]

The new version of geobr v1.6.5 that fxes this issue is now on CRAN.

[rafapereirabr]

Hello everyone and hello @rafapereirabr,

I tested from my side (Python - MacOS) and works. I only got that as a 'warning'

ShapelyDeprecationWarning: __len__ for multi-part geometries is deprecated and will be removed in Shapely 2.0. Check the length of thegeomsproperty instead to get the number of parts of a multi-part geometry.

Thank you for the effort to fix the issue. I hope that this 'warning' will not block us in the future ;-)

Best regards,

@luiz-dejesus, what function did you use that returned this 'warning' ?

[luiz-dejesus]

Hello @rafapereirabr, These are the steps that I did:

Best regards,

Luiz

[rafapereirabr]

I believe this issue has now been completely solved on both the R and Python version of geobr. I'm closing this issue for now, but please feel free to open a new issue if the problem persists.