csarven / worldbank-linkeddata

World Bank Linked Data
http://worldbank.270a.info/
Other
9 stars 9 forks source link

Resource not reachable #2

Open harsh9t opened 6 years ago

harsh9t commented 6 years ago

The resources at http://worldbank.270a.info/ are no longer reachable. Can this be fixed? If not is there another mirror of these datasets?

csarven commented 6 years ago

Yes, it can be fixed. Unfortunately, not soon. I can't recall exact mirrors if any out there.

The data was on a server that I no longer have access to. So, we'll have to rebuild / rerun the scripts on a new server. Something I'm looking forward to doing but can't get to it in the next months (a small part of my thesis needs this, so I might have to revisit - getting *.270a.info [WB in particular] going any way)

The dumps were at: http://worldbank.270a.info/data/

See also:

http://lod.openlinksw.com/ may (should?) had/has the data in there.

There may be an HDT around somewhere too.

Aside: Perhaps you can raise this in public-lod@w3.org for some archeological digging? :)

harsh9t commented 6 years ago

Thanks for the detailed reply.

If I wanted to build/ run the scripts myself, could you recommend or point me to some documentation (readme or something) which I could refer to do so? :)

csarven commented 6 years ago

I can't track worldbank in LOD Cache, but possibly of interest via @kidehen :

What we have in LOD Cache:

SPARQL Query Template:

SELECT DISTINCT  ?o
WHERE {
GRAPH <http://{dataset-origin}/data/import/> {?s a ?o .}
} 
LIMIT 100

Example:

SELECT DISTINCT  ?o
WHERE {
        GRAPH <http://fao.270a.info/data/import/> {?s a ?o .}
        # FILTER (CONTAINS(STR(?g),'270a.info'))
      } 
LIMIT 100

Origins: abs.270a.info bis.270a.info fao.270a.info imf.270a.info uis.270a.info bfs.270a.info ecb.270a.info frb.270a.info oecd.270a.info

Live Query Definition Example: https://tinyurl.com/ycxz9gmm

Live Query Result Example: https://tinyurl.com/y7k4qzh7

csarven commented 6 years ago

I think the scripts are fairly readable but unfortunately documentation is lacking. It was a long time ago :S (not an excuse). So, I'll have to revisit everything myself to make sense of it and document.

On a related note, we'll have to check the worldbank's APIs to see in what state they are in, because even if we have the scripts all ready to go, it might not work out in the end. The 4 major datasets: indicators, finances, projects, and climate, all used different endpoints/APIs to retrieve. We literally have to hunt them down, and possibly update the scripts. Most people are interested in the indicators, and I think WB finally has an SDMX endpoint nowadays. If the datasets they are publishing (somewhere up to 10k if I remember correctly) can be programmatically called, we can use https://github.com/csarven/linked-sdmx instead of WorldBank.java in this repo. That was always desired but when I worked on these datasets, they weren't using SDMX for their indicators datasets.

I can also check some personal backups elsewhere with small hopes of finding them, but I'm not counting on it since the datasets were quite large.