search

rOpenGov / eurostat

R tools for Eurostat data
http://ropengov.github.io/eurostat
Other
234 stars 46 forks source link

Error in parse_con(txt, bigint_as_char). lexical error: invalid char in json text. #302

Closed umbe1987 closed 4 months ago

umbe1987 commented 4 months ago

For some reasons, the same code that used to work until this morning is giving me now this error:

Error in parse_con(txt, bigint_as_char) : 
  lexical error: invalid char in json text.
                                       <html lang="en" class="no-js"> 
                     (right here) ------^

What I am doing:

library(eurostat)

clean_eurostat_cache()
get_eurostat(
  id = "migr_asyappctzm",
  filters = list(
    geo = c("BE","BG"),
    sex = c("T"),
    age = c("TOTAL"),
    asyl_app = c("NASY_APP"),
    citizen = "EXT_EU27_2020"
  ))
umbe1987 commented 4 months ago

I think Eurostat's website is having some problems right now, this might be the cause of the issue I am having...

https://ec.europa.eu/eurostat/databrowser/view/migr_asyappctzm/default/table?lang=en

image

umbe1987 commented 4 months ago

Yep, confirmed, closing...

image

umbe1987 commented 4 months ago

no wait, sorry for the confusion

it should only be about the folder “International trade in goods - detailed data (ext_go_detail)”, which is not the one I am using... Reopening for now.

I can add this: if I simply do get_eurostat(id = "migr_asyappctzm"), I receive

trying URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz'
Error in utils::download.file(url, tfile) : 
  cannot open URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz'
In addition: Warning message:
In utils::download.file(url, tfile) :
  cannot open URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz': HTTP status was '410 Gone'
Error in get_eurostat(id = "migr_asyappctzm") : 
  get_eurostat_raw fails with the id migr_asyappctzm

So the problem seems to be related to HTTP status was '410 Gone' and to API - Migrating from Bulk Download Listing urls to API urls.

pitkant commented 4 months ago

I think "web services / API" may refer to JSON web service and SDMX API, which both seem to be down at the moment. Not only with migr_asyappctzm but with other datasets as well.

umbe1987 commented 4 months ago

I just wrote to Eurostat support centre about this. Pretty sure it's a temporary issue, but will try to keep you posted about it.

pitkant commented 4 months ago

Thanks! Maybe adding a link to Eurostat Maintenance information would be helpful in some cases: https://ec.europa.eu/eurostat/web/main/help/maintenance-information

pitkant commented 4 months ago

no wait, sorry for the confusion

it should only be about the folder “International trade in goods - detailed data (ext_go_detail)”, which is not the one I am using... Reopening for now.

I can add this: if I simply do get_eurostat(id = "migr_asyappctzm"), I receive

trying URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz'
Error in utils::download.file(url, tfile) : 
  cannot open URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz'
In addition: Warning message:
In utils::download.file(url, tfile) :
  cannot open URL 'https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fmigr_asyappctzm.tsv.gz': HTTP status was '410 Gone'
Error in get_eurostat(id = "migr_asyappctzm") : 
  get_eurostat_raw fails with the id migr_asyappctzm

So the problem seems to be related to HTTP status was '410 Gone' and to API - Migrating from Bulk Download Listing urls to API urls.

Related to this issue: You might want to update your eurostat package to newest available version in CRAN. "estat-navtree-portlet-prod/BulkDownloadListing?" style links do not work any more. You can check your eurostat version with

packageVersion("eurostat")

and if it's not 4.0.0 then you have the old version.

umbe1987 commented 4 months ago

I had 3.8.2., just updated to 4.0.0. Thanks! P.S. just had to replace calls to eurotime2date2 with eurotime2date

umbe1987 commented 4 months ago

Just wanted to let you know that the problem still persists and that the page https://ec.europa.eu/eurostat/web/main/help/maintenance-information got an updated, and is no more 12/16 but 12/18 :crying_cat_face:

umbe1987 commented 4 months ago

Closing (everything back to normal)