search

dr-leo / pandaSDMX

Python interface to SDMX
Apache License 2.0
127 stars 58 forks source link

SDMX 2.0 issue maybe? #33

Closed Peter98 closed 8 years ago

Peter98 commented 8 years ago

Hi Dr Leo, Great work in concept, execution and documentation and I am really excited by what I can do with this tool. I have encountered a problem that might be an issue with SDMX 2.0 though a work around or any support would be greatly appreciated. Here's the code: r=Request() response=r.get(url='http://stat.abs.gov.au/restsdmx/sdmx.ashx/GetData/HF/2.0.99.10.132_3.M/ABS') and the error: ValueError: Unsupported root tag: {http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message}MessageGroup Happy to help and providers as I get seek more data sources. Thanks in Advance

dr-leo commented 8 years ago

Hi Peter98,

many thanks for your praise.

You've hit the nail on its head: SDMX 2.0 is not supported for the following reasons:

While it is not a priority for me to add 2.0 support, feel free to work on it if you see merit. If you manage to add a reader for this which somehow goes along with the SDMX 2.1 information model, I think it could be merged. That said, such an effort could be frustrating to the extent that agencies upgrade to SDMX 2.1. Note that IMF and OECD support 2.0-compliant metadata and 2.1-compliant data sets in JSON rather than XML.

If you are interested in data from that Australian agency, you may want to try quandl.

Any help to add a json reader thus opening the door to OECD and IMF data would be much appreciated. I am about to play around with jsonpath_rw. But I do not expect rapid progress as I have loads of other other obligations now.

Regards

Leo

Am 13.05.2016 um 16:03 schrieb Peter98:

Hi Dr Leo, Great work in concept, execution and documentation and I am really excited by what I can do with this tool. I have encountered a problem that might be an issue with SDMX 2.0 though a work around or any support would be greatly appreciated. Here's the code: r=Request() response=r.get(url='http://stat.abs.gov.au/restsdmx/sdmx.ashx/GetData/HF/2.0.99.10.132_3.M/ABS') and the error: ValueError: Unsupported root tag: {http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message}MessageGroup http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message%7DMessageGroup Happy to help and providers as I get seek more data sources. Thanks in Advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/dr-leo/pandaSDMX/issues/33

Peter98 commented 8 years ago

Thanks for the reply Dr Leo, I took a look at Quandl and it has a direct python API which is nice and will be a great source for international data, the Australian Agency (ABS) data is there but hard to find and not really user friendly, but I stumbled across an R project call rsdmx which works really nicely with the ABS data format which is SDMX .stat (dotstat). I love python but am pretty tool agnostic and building on this current project (pandaSDMX) at least for now is too much of a divergence from my real passion which is large scale big compute modelling and simulations on GPGPU and multi core multi machine cloud platforms. That being said if I can add anything here in the future I shall do so. Thanks again, Peter

dr-leo commented 8 years ago

Close.

Sounds interesting. - Wish I was that tool-agnostic...

Cheers, Leo

Am 14.05.2016 um 14:26 schrieb Peter98:

Thanks for the reply Dr Leo, I took a look at Quandl and it has a direct python API which is nice and will be a great source for international data, the Australian Agency (ABS) data is there but hard to find and not really user friendly, but I stumbled across an R project call rsdmx which works really nicely with the ABS data format which is SDMX .stat (dotstat). I love python but am pretty tool agnostic and building on this current project (pandaSDMX) at least for now is too much of a divergence from my real passion which is large scale big compute modelling and simulations on GPGPU and multi core multi machine cloud platforms. That being said if I can add anything here in the future I shall do so. Thanks again, Peter

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/dr-leo/pandaSDMX/issues/33#issuecomment-219217680

chris-beer commented 8 years ago

Hi Peter98

"If you are interested in data from that Australian agency, you may want to try quandl."

Alternatively - please do feel free to contact us directly at abs.stat@abs.gov.au for assistance with, or questions about, our SDMX data holdings and we'd be more than happy to assist.

(@ dr-leo - btw - we are aiming to be on SDMX-JSON and 2.1 at ABS.Stat within the next 3-4 months.)

Cheers

Chris

Chris Beer Lead (Data Services & API's) - Dissemination Enterprise Wide Services Statistical Business Transformation Program | Australian Bureau of Statistics   chris.beer@abs.gov.au     www.abs.gov.au

Peter98 commented 8 years ago

Hi Chris,

This is great news I would rather stay in the Python eco system even though the R tool rsdmx is excellent.

I spent the weekend playing around with ABS.Stat and its great having access to all that data natively in a data analysis tool, opens up a whole new world of modeling for me.

Will provide feedback through that email address if I find anything noteworthy.

Thanks to you and Dr Leo,

Peter

chris-beer commented 8 years ago

:)

No problems - on my side of the fence, I'm looking after the upgrade to .Stat (including making it 2.1), and I'm also the SDMX Technical Working Group contact for ABS - contact me directly below as needed.. You can see how it will work by looking at the OECD version (already upgraded) at stats.oecd.org.

If you are looking at the current version - the email address I supplied is the one to use.

Cheers

Chris

dr-leo commented 8 years ago

Have recently commited the json reader. Works for oecd data sets.

Am 14.05.2016 um 17:38 schrieb Dr. Leo fhaxbox66@googlemail.com:

Close.

Sounds interesting. - Wish I was that tool-agnostic...

Cheers, Leo

Am 14.05.2016 um 14:26 schrieb Peter98: Thanks for the reply Dr Leo, I took a look at Quandl and it has a direct python API which is nice and will be a great source for international data, the Australian Agency (ABS) data is there but hard to find and not really user friendly, but I stumbled across an R project call rsdmx which works really nicely with the ABS data format which is SDMX .stat (dotstat). I love python but am pretty tool agnostic and building on this current project (pandaSDMX) at least for now is too much of a divergence from my real passion which is large scale big compute modelling and simulations on GPGPU and multi core multi machine cloud platforms. That being said if I can add anything here in the future I shall do so. Thanks again, Peter

— You are receiving this because you commented. Reply to this email directly or view it on GitHub