search

rOpenGov / giscoR

Download geospatial data from GISCO API - Eurostat
https://ropengov.github.io/giscoR/
GNU General Public License v3.0
71 stars 1 forks source link

gisco_get_nuts(nuts_level = 1) can't no more resolve at resolutions "01" and "03" #89

Closed RemiDumas closed 2 weeks ago

RemiDumas commented 2 months ago 
> gisco_get_nuts(nuts_level = 1, resolution = "03")
HTTP Status Code: 503 - Service Unavailable

url 
  https://gisco-services.ec.europa.eu/distribution/v2/nuts/geojson/NUTS_RG_03M_2016_4326_LEVL_1.geojson  not reachable.

Please download manually.  If you think this is a bug please consider opening an issue on  https://github.com/ropengov/giscoR/issues
Returning `NULL`
NULL

It works with resolution "10", "20" and "60".

Last week, everything were fine

> sessionInfo()
R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 11 x64 (build 22631)

Matrix products: default

locale:
[1] LC_COLLATE=French_France.utf8  LC_CTYPE=French_France.utf8    LC_MONETARY=French_France.utf8 LC_NUMERIC=C                  
[5] LC_TIME=French_France.utf8    

time zone: Europe/Paris
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] sf_1.0-15       giscoR_0.5.0    scales_1.3.0    lubridate_1.9.2 forcats_1.0.0   stringr_1.5.1   dplyr_1.1.3     purrr_1.0.2    
 [9] readr_2.1.4     tidyr_1.3.0     tibble_3.2.1    ggplot2_3.4.3   tidyverse_2.0.0

loaded via a namespace (and not attached):
 [1] s2_1.1.6           rappdirs_0.3.3     utf8_1.2.3         generics_0.1.3     class_7.3-22       KernSmooth_2.23-21 stringi_1.8.3     
 [8] hms_1.1.3          magrittr_2.0.3     countrycode_1.6.0  grid_4.3.1         timechange_0.2.0   e1071_1.7-13       DBI_1.1.3         
[15] fansi_1.0.4        httr2_1.0.0        cli_3.6.1          rlang_1.1.1        units_0.8-5        munsell_0.5.0      withr_2.5.2       
[22] tools_4.3.1        tzdb_0.4.0         colorspace_2.1-0   geojsonsf_2.0.3    curl_5.2.0         vctrs_0.6.3        R6_2.5.1          
[29] proxy_0.4-27       lifecycle_1.0.4    classInt_0.4-10    pkgconfig_2.0.3    pillar_1.9.0       gtable_0.3.4       glue_1.6.2        
[36] Rcpp_1.0.11        xfun_0.42          tidyselect_1.2.0   rstudioapi_0.15.0  knitr_1.45         farver_2.1.1       labeling_0.4.3    
[43] wk_0.9.1           compiler_4.3.1
RemiDumas commented 2 months ago

Since this morning, gisco_get_nuts(nuts_level = 3, resolution = "10") doesn't work no more

dieghernan commented 2 months ago

Hi,

Thanks for reporting, it seems to be a temporary issue on the server side (not much to fix on this package). Mine is working in Windows (see reprex at the end of this post).

We saw some intermmitent issues from time to time (label not-downloading) that eventually are fixed, my best advice is to set a persistent cache_dir (see https://ropengov.github.io/giscoR/reference/gisco_set_cache_dir.html) with install = TRUE so next time you call the function it would use the cached geojson.

library(giscoR)

test_nuts1_03 <- gisco_get_nuts(nuts_level = 1, resolution = "03", 
                                cache_dir = tempdir())

head(test_nuts1_03)
#> Simple feature collection with 6 features and 10 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: 2.846719 ymin: 39.64579 xmax: 21.05607 ymax: 50.91189
#> Geodetic CRS:  WGS 84
#>   MOUNT_TYPE                                                   NAME_LATN
#> 1          0                                                   SHQIPËRIA
#> 2          0                                               OSTÖSTERREICH
#> 3          0                                               SÜDÖSTERREICH
#> 4          0 RÉGION DE BRUXELLES-CAPITALE/BRUSSELS HOOFDSTEDELIJK GEWEST
#> 5          0                                             RÉGION WALLONNE
#> 6          0                                              WESTÖSTERREICH
#>   CNTR_CODE                                                   NUTS_NAME
#> 1        AL                                                   SHQIPËRIA
#> 2        AT                                               OSTÖSTERREICH
#> 3        AT                                               SÜDÖSTERREICH
#> 4        BE RÉGION DE BRUXELLES-CAPITALE/BRUSSELS HOOFDSTEDELIJK GEWEST
#> 5        BE                                             RÉGION WALLONNE
#> 6        AT                                              WESTÖSTERREICH
#>   LEVL_CODE NUTS_ID URBN_TYPE COAST_TYPE FID geo                       geometry
#> 1         1     AL0         0          0 AL0 AL0 MULTIPOLYGON (((19.77166 42...
#> 2         1     AT1         0          0 AT1 AT1 MULTIPOLYGON (((15.51265 48...
#> 3         1     AT2         0          0 AT2 AT2 MULTIPOLYGON (((15.85203 47...
#> 4         1     BE1         0          0 BE1 BE1 MULTIPOLYGON (((4.459209 50...
#> 5         1     BE3         0          0 BE3 BE3 MULTIPOLYGON (((5.69828 50....
#> 6         1     AT3         0          0 AT3 AT3 MULTIPOLYGON (((14.66538 48...

test_nuts3_10 <- gisco_get_nuts(nuts_level = 3, resolution = "10", 
                                cache_dir = tempdir())

head(test_nuts3_10)
#> Simple feature collection with 6 features and 10 fields
#> Geometry type: MULTIPOLYGON
#> Dimension:     XY
#> Bounding box:  xmin: 23.96343 ymin: 42.71265 xmax: 27.01615 ymax: 44.03716
#> Geodetic CRS:  WGS 84
#>   MOUNT_TYPE      NAME_LATN CNTR_CODE
#> 1          4         Pleven        BG
#> 2          3         Lovech        BG
#> 3          4 Veliko Tarnovo        BG
#> 4          3        Gabrovo        BG
#> 5          4           Ruse        BG
#> 6          4        Razgrad        BG
#>                                                                                                   NUTS_NAME
#> 1                                                          <U+041F><U+043B><U+0435><U+0432><U+0435><U+043D>
#> 2                                                                  <U+041B><U+043E><U+0432><U+0435><U+0447>
#> 3 <U+0412><U+0435><U+043B><U+0438><U+043A><U+043E> <U+0422><U+044A><U+0440><U+043D><U+043E><U+0432><U+043E>
#> 4                                                  <U+0413><U+0430><U+0431><U+0440><U+043E><U+0432><U+043E>
#> 5                                                                          <U+0420><U+0443><U+0441><U+0435>
#> 6                                                  <U+0420><U+0430><U+0437><U+0433><U+0440><U+0430><U+0434>
#>   LEVL_CODE NUTS_ID URBN_TYPE COAST_TYPE   FID   geo
#> 1         3   BG314         2          3 BG314 BG314
#> 2         3   BG315         2          3 BG315 BG315
#> 3         3   BG321         2          3 BG321 BG321
#> 4         3   BG322         2          3 BG322 BG322
#> 5         3   BG323         2          3 BG323 BG323
#> 6         3   BG324         3          3 BG324 BG324
#>                         geometry
#> 1 MULTIPOLYGON (((24.50009 43...
#> 2 MULTIPOLYGON (((25.10645 43...
#> 3 MULTIPOLYGON (((25.39146 43...
#> 4 MULTIPOLYGON (((24.97323 43...
#> 5 MULTIPOLYGON (((26.16028 43...
#> 6 MULTIPOLYGON (((26.83469 43...

Created on 2024-07-04 with reprex v2.1.0

dieghernan commented 2 months ago

Ping @hannesaddec (GISCO)

hannesaddec commented 2 months ago

All services are nominal. For investigation we would need the underlying error message from the URL request in R ?

dieghernan commented 2 months ago

Hi @hannesaddec , what I got is a Error 503 Backend fetch failed.

library(httr2)

req <- httr2::request("https://gisco-services.ec.europa.eu/distribution/v2/nuts/geojson/NUTS_RG_03M_2016_4326_LEVL_1.geojson")

resp <- req |>
  req_error(is_error = \(resp) FALSE) |>
  req_perform()

resp_body_string(resp)
#> [1] "<!DOCTYPE html>\n<html>\n  <head>\n    <title>503 Backend fetch failed</title>\n  </head>\n  <body>\n    <h1>Error 503 Backend fetch failed</h1>\n    <p>Backend fetch failed</p>\n    <h3>Guru Meditation:</h3>\n    <p>XID: 298292587</p>\n    <hr>\n    <p>Varnish cache server</p>\n  </body>\n</html>\n"
<html>
  <head>
      <title>503 Backend fetch failed</title>
   </head>
   <body>
      <h1>Error 503 Backend fetch failed</h1>
      <p>Backend fetch failed</p>
      <h3>Guru Meditation:</h3>
      <p>XID: 298292587</p>
      <hr>
      <p>Varnish cache server</p>
   </body>
</html>

Same result with curl commands, see reqbin here: https://reqbin.com/c-olhq6tjw

Created on 2024-07-05 with reprex v2.1.0

hannesaddec commented 2 months ago

should be fixed.. next time please add IP address/time via PM

dieghernan commented 2 months ago

Would do, thanks