search

rOpenGov / eurostat

R tools for Eurostat data
http://ropengov.github.io/eurostat
Other
234 stars 46 forks source link

fails with id "proj_19rp3" #304

Closed JensXII closed 4 months ago

JensXII commented 4 months ago

Cannot download regional population projection data:

X <- eurostat::get_eurostat("proj_19rp3") Error in eurostat::get_eurostat("proj_19rp3") : get_eurostat_raw fails with the id proj_19rp3

It worked a few weeks ago.

Best regards Jens

pitkant commented 4 months ago

Thank you for reporting this @JensXII

This is related to #303 and to the discussion under #293 . The problem is related to Eurostat returning an ASYNC API response to data queries at a lower threshold than before, see documentation 1 2

We're currently working on finding some kind of intermediate solution to this problem. In the mean time using filters to make your query smaller might work.

pitkant commented 4 months ago

I have received the following message from Eurostat user support:

"Last week maintenance introduced major changes to internal storage that required a big step forward and to renew the cache from scratch.

Sadly this release also contained a performance issue that remained undetected that is currently slowing down the repopulation process.

A hotfix was applied today at around 13:30."

So I think that the issues are related to big datasets not being cached as they previously were. Excerpt from the Eurostat documentation:

"When a data request is initiated, the system first checks if the exact same request was already performed previously and if applicable lookup the data directly from an internal cache and return it as a response."

I'm not sure if today's hotfix has renewed the cache for all files or not (probably not, sounds like a process that takes some time) but maybe something has changed for the better now.

JensXII commented 4 months ago

Thanks - I works for me now :-)