search

pydata / pandas-datareader

Extract data from a wide range of Internet sources into a pandas DataFrame.
https://pydata.github.io/pandas-datareader/stable/index.html
Other
2.93k stars 681 forks source link

read json-stat files #200

Open hmelberg opened 8 years ago

hmelberg commented 8 years ago

Several international agencies use the json-stat format. Would there be any interest in making datareader/pandas easily read this format?

I can help, but will probably need some need help helping!

By the way: According to the json-stat website the list of agencies using json-stat includes: National Institute for Statistics of Italy, Statistics Norway, UK’s Office for National Statistics, Statistics Sweden, Statistics Denmark, Statistics Greenland, Statistics Finland, Instituto Galego de Estatística, Central Statistics Office of Ireland, United Nations Economic Commission for Europe, Eurostat

femtotrader commented 8 years ago

Interesting idea.

Here is JSON-stat spec https://json-stat.org/

and some JS tools https://github.com/badosa/JSON-stat/tree/master/utils

some other tools (including Python) can be found at https://json-stat.org/tools/

https://github.com/predicador37/pyjstat https://github.com/26fe/jsonstat.py

Pinging @predicador37 @26fe

hmelberg commented 8 years ago

I have made an attempt to create a tool that allows the user to easily download data from Statistics Norway to Pandas:

https://gist.github.com/anonymous/e1463d45e4c4e8673bfcfbaf585cdd8c

Perhaps, with some revision, some of this could be used by datareader? Comments are welcome.

One key issue is that in order to read a json-stat file, one needs to specify the exact query for which variables/values to be included and so on. This is sometimes quite complicated, and to make this easier for beginner I included a widget that generates the json-stat query based on the users input in a gui. This cannot be included in datareader, I guess, but the function for just reading a full table with all information, may fit in this project.

If this looks interesting, I may try to create a PR.

sinhrks commented 8 years ago

I once wrote a func to do it. Willing to move it under pandas_datareader.io